Skip to content

Redis online store fails to maintain event-time ordering in batch writes #5163

Description

@sundeep-sf

Expected Behavior

When writing a batch of events to Redis online store, the store should maintain event-time ordering guarantees regardless of the order of events within the batch. The value associated with the latest timestamp should be preserved, even if it appears earlier in the batch.

Current Behavior

The Redis online store processes events in batch order rather than timestamp order. When writing a batch containing events in reverse chronological order (e.g., [t3, t2, t1]), it incorrectly skips later events because it compares timestamps sequentially, causing the wrong values to be stored.

Steps to reproduce

t1 = datetime.now(timezone.utc)
t2 = t1 + timedelta(seconds=5)
t3 = t2 + timedelta(seconds=5)

# Create batch with reverse chronological order
test_data = pd.DataFrame({
    "customer_id": [1] * 3,
    "total_purchases": [30, 20, 10],
    "avg_order_value": [300.0, 200.0, 100.0],
    "event_timestamp": [t3, t2, t1]  # Latest timestamp first
})

# Push batch to online store
feature_store.push_features(feature_group_name, test_data, mode="online")

# Retrieve feature - will return value 10 instead of expected 30
result = feature_store.get_online_features(...)

Specifications

  • Version: Feast SDK Version: "0.40.1"
  • Platform: Any
  • Subsystem: Redis Online Store

Possible Solution

In online_write_batch, before processing the batch:
Sort the data list by timestamp in ascending order, OR
Find the latest timestamp for each entity key and only write that record
The current implementation in online_write_batch processes records sequentially and skips records with timestamps older than what's already been processed in the current batch, which leads to incorrect behavior when events are not in chronological order.
Relevant code: https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/online_stores/redis.py#L582

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions