Expected Behavior
feast apply against a HybridOnlineStore should route every FeatureView to its configured backend, using the tag named by routing_tag.
Current Behavior
It works for the first FeatureView and then fails for the second, regardless of how the views are tagged:
Applying changes for project mantis
Traceback (most recent call last):
...
File ".../feast/infra/online_stores/hybrid_online_store/hybrid_online_store.py", line 273, in update
raise ValueError(
ValueError: FeatureView must have a 'tribe' tag to use HybridOnlineStore.
Note the message says 'tribe' even though the repo config sets routing_tag: online_store_dev. That string is not hardcoded — it is the configured value with "tribe" as the fallback:
tag_name = getattr(config.online_store, "routing_tag", "tribe")
Getting the fallback means config.online_store is no longer the HybridOnlineStoreConfig by the time the second view is processed.
Root cause
Two things combine.
1. _prepare_repo_conf mutates the caller's RepoConfig in place. rconfig = config is an alias, not a copy, so both the attribute assignment and the __dict__ writes land on the caller's object — replacing online_store with the selected backend's config. It also injects type into online_store.conf, which is the user's own dict.
def _prepare_repo_conf(self, config: RepoConfig, online_store_type: str):
rconfig = config # alias, not a copy
for online_store in config.online_store.online_stores:
if online_store.type.split(".")[-1].lower() == online_store_type.lower():
rconfig.online_config = online_store.conf
rconfig.online_config["type"] = online_store.type # mutates caller's conf
data = rconfig.__dict__ # caller's __dict__
data["registry"] = data["registry_config"]
data["offline_store"] = data["offline_config"]
data["online_store"] = data["online_config"] # hybrid config overwritten
return data
2. update() rebinds config inside the loop over tables_to_keep, so even a non-mutating _prepare_repo_conf would feed the next iteration the narrowed config:
for table in tables_to_keep:
tribe = self._get_routing_tag_value(table, config) # config from the previous iteration
if not tribe:
raise ValueError(...)
...
config = RepoConfig(**self._prepare_repo_conf(config, tribe))
teardown() has the same rebinding inside its loop. online_write_batch() and online_read() rebind too; those are single-use per call, but they still leave the caller's config mutated through (1).
This makes HybridOnlineStore effectively unusable for any repo with more than one FeatureView.
Steps to reproduce
- Configure a
HybridOnlineStore with two backends and a custom routing_tag:
online_store:
type: hybrid
routing_tag: backend
online_stores:
- type: redis
conf: {redis_type: redis, connection_string: "localhost:6379"}
- type: sqlite
conf: {path: "/tmp/feast_hybrid_test.db"}
- Define two
FeatureViews, one tagged backend: redis and one tagged backend: sqlite.
- Run
feast apply.
The first view is applied; the second raises ValueError: FeatureView must have a 'tribe' tag to use HybridOnlineStore.
Equivalent as a test (fails on master, passes with the fix):
HybridOnlineStore().update(
config=repo_config,
tables_to_delete=[],
tables_to_keep=[fv_redis, fv_sqlite],
entities_to_delete=[],
entities_to_keep=[entity],
partial=False,
)
Specifications
- Version: 0.65.0 (also present on
master as of this writing)
- Platform: Linux / Python 3.12, and reproduced locally on Python 3.11
- Subsystem:
feast.infra.online_stores.hybrid_online_store
Possible Solution
- Build the returned mapping from a copy in
_prepare_repo_conf, and construct a new dict for the backend conf instead of injecting type into the caller's.
- Assign the per-backend
RepoConfig to a local name in update() and teardown() rather than rebinding the config parameter inside the loop.
Happy to open a PR with exactly this plus a unit regression test — I have it working locally.
Expected Behavior
feast applyagainst aHybridOnlineStoreshould route everyFeatureViewto its configured backend, using the tag named byrouting_tag.Current Behavior
It works for the first
FeatureViewand then fails for the second, regardless of how the views are tagged:Note the message says
'tribe'even though the repo config setsrouting_tag: online_store_dev. That string is not hardcoded — it is the configured value with"tribe"as the fallback:Getting the fallback means
config.online_storeis no longer theHybridOnlineStoreConfigby the time the second view is processed.Root cause
Two things combine.
1.
_prepare_repo_confmutates the caller'sRepoConfigin place.rconfig = configis an alias, not a copy, so both the attribute assignment and the__dict__writes land on the caller's object — replacingonline_storewith the selected backend's config. It also injectstypeintoonline_store.conf, which is the user's own dict.2.
update()rebindsconfiginside the loop overtables_to_keep, so even a non-mutating_prepare_repo_confwould feed the next iteration the narrowed config:teardown()has the same rebinding inside its loop.online_write_batch()andonline_read()rebind too; those are single-use per call, but they still leave the caller's config mutated through (1).This makes
HybridOnlineStoreeffectively unusable for any repo with more than oneFeatureView.Steps to reproduce
HybridOnlineStorewith two backends and a customrouting_tag:FeatureViews, one taggedbackend: redisand one taggedbackend: sqlite.feast apply.The first view is applied; the second raises
ValueError: FeatureView must have a 'tribe' tag to use HybridOnlineStore.Equivalent as a test (fails on master, passes with the fix):
Specifications
masteras of this writing)feast.infra.online_stores.hybrid_online_storePossible Solution
_prepare_repo_conf, and construct a new dict for the backend conf instead of injectingtypeinto the caller's.RepoConfigto a local name inupdate()andteardown()rather than rebinding theconfigparameter inside the loop.Happy to open a PR with exactly this plus a unit regression test — I have it working locally.