Skip to content

HybridOnlineStore loses routing_tag after the first FeatureView, so feast apply fails #6768

Description

@jang-hs

Expected Behavior

feast apply against a HybridOnlineStore should route every FeatureView to its configured backend, using the tag named by routing_tag.

Current Behavior

It works for the first FeatureView and then fails for the second, regardless of how the views are tagged:

Applying changes for project mantis
Traceback (most recent call last):
  ...
  File ".../feast/infra/online_stores/hybrid_online_store/hybrid_online_store.py", line 273, in update
    raise ValueError(
ValueError: FeatureView must have a 'tribe' tag to use HybridOnlineStore.

Note the message says 'tribe' even though the repo config sets routing_tag: online_store_dev. That string is not hardcoded — it is the configured value with "tribe" as the fallback:

tag_name = getattr(config.online_store, "routing_tag", "tribe")

Getting the fallback means config.online_store is no longer the HybridOnlineStoreConfig by the time the second view is processed.

Root cause

Two things combine.

1. _prepare_repo_conf mutates the caller's RepoConfig in place. rconfig = config is an alias, not a copy, so both the attribute assignment and the __dict__ writes land on the caller's object — replacing online_store with the selected backend's config. It also injects type into online_store.conf, which is the user's own dict.

def _prepare_repo_conf(self, config: RepoConfig, online_store_type: str):
    rconfig = config                                   # alias, not a copy
    for online_store in config.online_store.online_stores:
        if online_store.type.split(".")[-1].lower() == online_store_type.lower():
            rconfig.online_config = online_store.conf
            rconfig.online_config["type"] = online_store.type   # mutates caller's conf
    data = rconfig.__dict__                            # caller's __dict__
    data["registry"] = data["registry_config"]
    data["offline_store"] = data["offline_config"]
    data["online_store"] = data["online_config"]       # hybrid config overwritten
    return data

2. update() rebinds config inside the loop over tables_to_keep, so even a non-mutating _prepare_repo_conf would feed the next iteration the narrowed config:

for table in tables_to_keep:
    tribe = self._get_routing_tag_value(table, config)   # config from the previous iteration
    if not tribe:
        raise ValueError(...)
    ...
    config = RepoConfig(**self._prepare_repo_conf(config, tribe))

teardown() has the same rebinding inside its loop. online_write_batch() and online_read() rebind too; those are single-use per call, but they still leave the caller's config mutated through (1).

This makes HybridOnlineStore effectively unusable for any repo with more than one FeatureView.

Steps to reproduce

  1. Configure a HybridOnlineStore with two backends and a custom routing_tag:
online_store:
  type: hybrid
  routing_tag: backend
  online_stores:
    - type: redis
      conf: {redis_type: redis, connection_string: "localhost:6379"}
    - type: sqlite
      conf: {path: "/tmp/feast_hybrid_test.db"}
  1. Define two FeatureViews, one tagged backend: redis and one tagged backend: sqlite.
  2. Run feast apply.

The first view is applied; the second raises ValueError: FeatureView must have a 'tribe' tag to use HybridOnlineStore.

Equivalent as a test (fails on master, passes with the fix):

HybridOnlineStore().update(
    config=repo_config,
    tables_to_delete=[],
    tables_to_keep=[fv_redis, fv_sqlite],
    entities_to_delete=[],
    entities_to_keep=[entity],
    partial=False,
)

Specifications

  • Version: 0.65.0 (also present on master as of this writing)
  • Platform: Linux / Python 3.12, and reproduced locally on Python 3.11
  • Subsystem: feast.infra.online_stores.hybrid_online_store

Possible Solution

  • Build the returned mapping from a copy in _prepare_repo_conf, and construct a new dict for the backend conf instead of injecting type into the caller's.
  • Assign the per-backend RepoConfig to a local name in update() and teardown() rather than rebinding the config parameter inside the loop.

Happy to open a PR with exactly this plus a unit regression test — I have it working locally.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions