## Is your feature request related to a problem? Please describe. Currently, when using the Feature Server's push API with streaming ingestion, offline store writes are not automatically batched. Users must either: 1. Manually batch calls to `write_to_offline_store()` in their application code 2. Set up a separate stream processing pipeline (e.g., Spark with Kafka) to handle batching For users receiving high-frequency feature updates (e.g., 100 requests/second) who want to write to both online and offline stores, this creates unnecessary complexity and can lead to inefficient writes to offline stores like BigQuery or Parquet. ## Describe the solution you'd like Add configurable automatic batching for offline store writes in the Feature Server, similar to how the Spark-Kafka processor handles batching via the `processingTime` parameter. **Proposed configuration options:** - `offline_write_batch_size`: Number of records to buffer before flushing to offline store - `offline_write_batch_interval`: Time interval (seconds) to wait before flushing to offline store - Flush on whichever threshold is reached first **Example configuration in `feature_store.yaml`:** ```yaml online_store: type: redis ... offline_store: type: bigquery ... feature_server: offline_write_batching: enabled: true batch_size: 1000 batch_interval_seconds: 30 ``` ## Describe alternatives you've considered 1. **Current approach with stream processor**: Users can set up a Spark Structured Streaming job that reads from Kafka and uses `processingTime` to control batching. This works but requires additional infrastructure and complexity. 2. **Manual batching in application code**: Users can implement their own buffering logic before calling `write_to_offline_store()`, but this duplicates effort across different applications. 3. **Use only online store**: Skip offline store writes entirely during streaming and rely on periodic materialization from batch sources, but this loses the real-time historical logging capability. ## Additional context This feature request came from a community discussion about streaming ingestion patterns. When using `PushMode.ONLINE_AND_OFFLINE` with high-frequency updates, the lack of automatic batching can result in: - Excessive API calls to offline stores (one per feature update) - Higher costs for cloud data warehouses like BigQuery - Increased latency for write operations **Current implementation reference:** The Spark-Kafka processor already implements batching at `sdk/python/feast/infra/contrib/spark_kafka_processor.py:134-173` using `foreachBatch`. Similar batching logic could be added to the Feature Server's push endpoint. **Benefits:** - Simplified architecture for users who want real-time serving + historical logging - Reduced costs for offline store writes - Better alignment with best practices for data warehouse ingestion - No breaking changes (feature would be opt-in via configuration) Related code: - Feature Server: `sdk/python/feast/feature_store.py:1786-1828` (`write_to_offline_store`) - Spark batching example: `sdk/python/feast/infra/contrib/spark_kafka_processor.py:134-173` cc @feast-dev/maintainers