Expected Behavior
When writing a batch of events to Redis online store, the store should maintain event-time ordering guarantees regardless of the order of events within the batch. The value associated with the latest timestamp should be preserved, even if it appears earlier in the batch.
Current Behavior
The Redis online store processes events in batch order rather than timestamp order. When writing a batch containing events in reverse chronological order (e.g., [t3, t2, t1]), it incorrectly skips later events because it compares timestamps sequentially, causing the wrong values to be stored.
Steps to reproduce
t1 = datetime.now(timezone.utc)
t2 = t1 + timedelta(seconds=5)
t3 = t2 + timedelta(seconds=5)
# Create batch with reverse chronological order
test_data = pd.DataFrame({
"customer_id": [1] * 3,
"total_purchases": [30, 20, 10],
"avg_order_value": [300.0, 200.0, 100.0],
"event_timestamp": [t3, t2, t1] # Latest timestamp first
})
# Push batch to online store
feature_store.push_features(feature_group_name, test_data, mode="online")
# Retrieve feature - will return value 10 instead of expected 30
result = feature_store.get_online_features(...)
Specifications
- Version:
Feast SDK Version: "0.40.1"
- Platform: Any
- Subsystem: Redis Online Store
Possible Solution
In online_write_batch, before processing the batch:
Sort the data list by timestamp in ascending order, OR
Find the latest timestamp for each entity key and only write that record
The current implementation in online_write_batch processes records sequentially and skips records with timestamps older than what's already been processed in the current batch, which leads to incorrect behavior when events are not in chronological order.
Relevant code: https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/online_stores/redis.py#L582
Expected Behavior
When writing a batch of events to Redis online store, the store should maintain event-time ordering guarantees regardless of the order of events within the batch. The value associated with the latest timestamp should be preserved, even if it appears earlier in the batch.
Current Behavior
The Redis online store processes events in batch order rather than timestamp order. When writing a batch containing events in reverse chronological order (e.g., [t3, t2, t1]), it incorrectly skips later events because it compares timestamps sequentially, causing the wrong values to be stored.
Steps to reproduce
Specifications
Feast SDK Version: "0.40.1"Possible Solution
In online_write_batch, before processing the batch:
Sort the data list by timestamp in ascending order, OR
Find the latest timestamp for each entity key and only write that record
The current implementation in online_write_batch processes records sequentially and skips records with timestamps older than what's already been processed in the current batch, which leads to incorrect behavior when events are not in chronological order.
Relevant code: https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/online_stores/redis.py#L582