Skip to content

feat: Added SQL registry schema_mode and registry create command - #6704

Merged
nquinn408 merged 4 commits into
feast-dev:masterfrom
AdityaPatil22:feat/feature-based-registry
Aug 9, 2026
Merged

feat: Added SQL registry schema_mode and registry create command#6704
nquinn408 merged 4 commits into
feast-dev:masterfrom
AdityaPatil22:feat/feature-based-registry

Conversation

@AdityaPatil22

@AdityaPatil22 AdityaPatil22 commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

Adds a schema_mode configuration option to the SQL registry that controls whether Feast creates database tables on startup. This addresses production environments where the application database user should not have DDL (CREATE TABLE) privileges.

Three modes are supported:

  • auto (default) - current behavior, creates tables if they don't exist. No breaking change.
  • verify - skips DDL but checks that all expected tables exist on startup. Raises a clear error listing missing tables with remediation instructions.
  • skip - skips both creation and verification. For environments where schema is fully managed externally (e.g. migration tools like Alembic/Flyway).

Also adds a feast registry create-schema CLI command to pre-create the schema with a privileged user, so the runtime user only needs DML access.

Which issue(s) this PR fixes:

Fixes #6678

Checks

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests
  • Testing is not required for this change

Unit tests - 5 new tests in TestSchemaMode covering: auto creates tables, verify raises on missing tables, verify passes with pre-created schema, skip works with existing schema, invalid values rejected by Pydantic.

Manual tests - Verified all three modes end-to-end against SQLite: auto creates and applies, verify succeeds with existing schema and fails without, skip works, east registry create-schema creates schema for verify/skip to use.

@AdityaPatil22
AdityaPatil22 requested a review from a team as a code owner August 5, 2026 08:10
@AdityaPatil22 AdityaPatil22 mentioned this pull request Aug 5, 2026
Closed
@codecov-commenter

codecov-commenter commented Aug 5, 2026 •
edited
Loading

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 67.39130% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.76%. Comparing base (b7ae488) to head (548ed0b).

Files with missing lines Patch % Lines
sdk/python/feast/cli/registry.py 43.47% 13 Missing ⚠️
sdk/python/feast/infra/registry/sql.py 90.47% 1 Missing and 1 partial ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6704      +/-   ##
==========================================
+ Coverage   46.75%   46.76%   +0.01%     
==========================================
  Files         414      415       +1     
  Lines       50348    50392      +44     
  Branches     7208     7214       +6     
==========================================
+ Hits        23539    23568      +29     
- Misses      25157    25171      +14     
- Partials     1652     1653       +1
Flag Coverage Δ
go-feature-server 30.58% <ø> (ø)
python-unit 48.09% <67.39%> (+0.01%) ⬆️
Files with missing lines Coverage Δ
sdk/python/feast/cli/cli.py 55.17% <100.00%> (+0.34%) ⬆️
sdk/python/feast/infra/registry/sql.py 58.21% <90.47%> (+0.84%) ⬆️
sdk/python/feast/cli/registry.py 43.47% <43.47%> (ø)

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b7ae488...548ed0b. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nquinn408

nquinn408 commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

@AdityaPatil22 Thanks for this — it's a focused, backward-compatible change that fills a real gap (running the SQL registry against a database where the runtime role shouldn't hold DDL grants). Overall looks good; a few minor points below, none blocking.

Strengths

  • Default auto preserves existing behavior — no migration needed for current users.
  • FeastRegistrySchemaError is actionable: it names the missing tables and the remediation command.
  • The CLI engine construction mirrors the write-engine build (path + sqlalchemy_config_kwargs), so the pre-created schema matches what the runtime expects.
  • Good docs coverage (yaml reference, CLI reference, dedicated SQL-registry section), and idempotency is called out.

Suggestions (minor / follow-up)

  1. verify checks table existence only, not columns. _verify_schema compares table-name sets. A schema created by an older Feast version (missing a newer column) would pass verify but fail at query time. _warn_if_narrow_blob_columns already inspects column metadata, so there's precedent — worth either a doc caveat that verify is table-level only, or a follow-up that also checks expected columns.
  2. verify skips the read replica. init only calls _verify_schema(self.write_engine). When read_path points at a separate replica, a missing table there won't be caught. The existing code runs _warn_if_narrow_blob_columns against both engines — mirroring that (verify the read engine too when read_engine is not write_engine) would be consistent.
  3. teardown() still issues DDL. The stated motivation is "the app doesn't need DDL privileges at runtime," but teardown() drops tables regardless of schema_mode. That's defensible since teardown is an explicit admin action, but a one-line doc note would prevent surprise for a verify/skip deployment whose runtime role genuinely lacks DROP.
  4. test_schema_mode_skip_does_not_run_ddl doesn't fully prove the claim. It pre-creates the schema, so it can't distinguish skip from auto/verify. A stronger assertion would run skip against an empty DB and assert it (a) does not raise and (b) leaves the DB with no registry tables — that directly proves "no DDL, no verification."

Nits

  • The new from sqlalchemy import (inspect as sa_inspect,) is a separate import block directly below the existing large from sqlalchemy import (...) — worth folding in.
  • FeastRegistrySchemaError.init(self, missing_tables: list[str]) uses list[str] while the rest of the file uses typing.List. Cosmetic, but matching the file convention would be nice.
  • The test tempfiles from mkstemp() are never removed — teardown() drops the tables but the sqlite file leaks in the temp dir. Consider cleanup or a tmp_path fixture.

nquinn408
nquinn408 reviewed Aug 5, 2026
View reviewed changes

@nquinn408 nquinn408 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See feedback

@AdityaPatil22

AdityaPatil22 commented Aug 5, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@nquinn408 Thanks for the thorough review!

Addressed all points:
verify checks table existence only, not columns - Added a doc caveat to the verify row in the mode table noting it's table-level only and that a schema from an older Feast version may pass verification but fail at query time. Column-level verification can be a follow-up.

verify skips the read replica - _verify_schema now runs against read_engine too when it differs from write_engine, mirroring the existing _warn_if_narrow_blob_columns pattern.

teardown() still issues DDL - On closer inspection, teardown() actually runs DELETE FROM (DML), not DROP TABLE (DDL), so it doesn't need DDL privileges. Updated the doc to correct this pre-existing inaccuracy: "the registry deletes all rows from the registry tables (it does not drop the tables themselves). This runs regardless of schema_mode and requires only DML (DELETE) privileges, not DDL."

test_schema_mode_skip_does_not_run_ddl - Rewritten to mock metadata.create_all and SqlRegistry._verify_schema, asserting neither is called. This directly proves the "no DDL, no verification" claim — the old test pre-created the schema so it couldn't distinguish skip from auto/verify.

Nits:

  • list[str] → List[str] to match the file's typing.List convention.
  • Folded inspect as sa_inspect into the main sqlalchemy block; ruff re-separates it because the alias can't share the # type: ignore comment - this is as merged as the linter allows.
  • All tests switched from tempfile.mkstemp() to pytest tmp_path fixture for automatic cleanup.

@AdityaPatil22
AdityaPatil22 requested a review from nquinn408 August 5, 2026 18:03
nquinn408
nquinn408 approved these changes Aug 5, 2026
View reviewed changes

@nquinn408 nquinn408 left a comment •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One residual consideration (non-blocking)

Adding _verify_schema(self.read_engine) means verify mode now hard-fails startup if the read replica is missing a table — even transiently. There's already a comment nearby noting "a read replica can be on a different schema version (e.g. mid-migration)." For verify that strictness is arguably the intent (you asked it to assert the schema), so I'd keep it — but it's worth the author being aware that a lagging replica mid-migration will now block startup in verify mode. A one-line doc mention would be nice; not required.

@AdityaPatil22
feat: Added SQL registry schema_mode and registry create command
eb61756 
Signed-off-by: Aditya Patil <adityapatil7649@gmail.com>
@AdityaPatil22
fix: Refine SQL registry schema_mode behavior and docs
50c9cf6 
Signed-off-by: Aditya Patil <adityapatil7649@gmail.com>
@AdityaPatil22
docs: Document read_path replica verification in SQL registry
d505e03 
Signed-off-by: Aditya Patil <adityapatil7649@gmail.com>
@AdityaPatil22
AdityaPatil22 force-pushed the feat/feature-based-registry branch from 1f64eee to d505e03 Compare August 6, 2026 05:44
@AdityaPatil22

AdityaPatil22 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

@nquinn408 - I have updated the doc as suggested

@nquinn408

nquinn408 commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

@AdityaPatil22 Where are we with this PR? There is a test failure. Thanks for your work on this!

@nquinn408
nquinn408 merged commit 037c4cd into feast-dev:master Aug 9, 2026
25 checks passed
franciscojavierarceo pushed a commit that referenced this pull request Aug 21, 2026
@feast-ci-bot
chore(release): release 0.66.0
1d5be95 
# [0.66.0](v0.65.0...v0.66.0) (2026-08-21)

### Bug Fixes

* Add connection pre-warming for DynamoDB async client ([89240fa](89240fa)), closes [#6060](#6060)
* Add remote registry client extra ([#6697](#6697)) ([b8dfcb0](b8dfcb0))
* Address review feedback on FIPS cipher suite configuration ([4a35fba](4a35fba))
* Allow remote-registry first apply for new projects ([39d408d](39d408d))
* Avoid importing feast.feature_store at mcp_server import time ([ddb2e9a](ddb2e9a))
* Bump pymssql to >=2.3.6 for macOS arm64 wheel support ([181eb35](181eb35)), closes [#5636](#5636) [#5193](#5193) [#5636](#5636)
* Call ApplySavedDataset RPC instead of ApplyFeatureService in RemoteRegistry.apply_saved_dataset() ([934d341](934d341))
* Catch missing dbt parser dependency in dbt CLI commands ([#6534](#6534)) ([3c2ae3c](3c2ae3c))
* Default authentication to kubernetes auth ([6a4690a](6a4690a))
* Defer feature-freshness thread to post-fork to avoid Gunicorn deadlock ([#6648](#6648)) ([104ad10](104ad10)), closes [#6647](#6647)
* Do not pass undeclared feature view columns to ODFV UDFs ([#6527](#6527)) ([75b9463](75b9463))
* downgrade mcp pin to 1.29.0 and fix CI lockfiles and unit tests ([98e5bca](98e5bca)), closes [#6706](#6706)
* Feast apply silently ignoring ttl updates to None or timedelta(0) ([#6709](#6709)) ([97b0f25](97b0f25)), closes [#6703](#6703)
* Fix mypy TorchTensor type alias error ([#6712](#6712)) ([34de6fa](34de6fa)), closes [#5563](#5563)
* Fixed data source creation form gaps ([5d0f7d6](5d0f7d6))
* Handle parameterized and complex Trino types in type map ([326554d](326554d))
* Isolate default user permissions ([e37adbf](e37adbf))
* Isolate projection join key maps ([d1c709d](d1c709d))
* Map Postgres real to FLOAT instead of DOUBLE ([62db435](62db435))
* Merge shared ODFV source projections in feature resolution ([d269946](d269946)), closes [#6621](#6621)
* More exhaustive athena types ([a9aaefc](a9aaefc))
* Normalize SQL registry read_path to the psycopg3 driver like path ([#6644](#6644)) ([996c6ea](996c6ea)), closes [#6643](#6643)
* **operator:** add spec.services.onlineStore.disabled to opt out of the online store ([d81d4e3](d81d4e3)), closes [#6586](#6586)
* Preinstall DuckDB delta extension for tests ([fd4d49d](fd4d49d)), closes [#6743](#6743)
* Preserve event-time ordering within Redis online_write_batch ([40fb788](40fb788)), closes [#5163](#5163)
* Prevent mutation of cached feature resolution results ([ea17419](ea17419))
* Remote feastRef FeatureStore fails first apply for a new feastProject ([9affee5](9affee5))
* Remove inert subjectaccessreviews and reorganize RBAC rules ([f771ea4](f771ea4))
* Report single-feature-view spark_application materialization success ([a9219d9](a9219d9)), closes [#6673](#6673)
* Reset the global security manager after the permissions fixture ([7667215](7667215))
* Resolve kserve with pip --dry-run instead of installing it ([01da132](01da132)), closes [#6732](#6732)
* Resolve write_to_offline_store feature view with a single registry lookup ([a42dc85](a42dc85)), closes [#4235](#4235)
* Return False from __eq__ on cross-type comparison ([#6637](#6637)) ([0f149a9](0f149a9)), closes [#6636](#6636)
* Reuse IdP-issued client tokens until near expiry ([602d752](602d752))
* Reuse the OIDC JWKS client across requests ([#6683](#6683)) ([a1e6fc2](a1e6fc2))
* Separate CronJob and feature-server ServiceAccounts ([398f643](398f643))
* Serialize UnixTimestamp proto values as raw int64 in remote online store transport ([1e7134f](1e7134f))
* Set FIPS cipher suites before pyarrow.flight import to prevent crash on IBM Power ([979b82a](979b82a))
* Support Entra ID (Azure AD) token claims in OIDC auth ([#6631](#6631)) ([f843c63](f843c63))
* UDF/ODFV source rehydrate (+ Postgres / online cache) ([#6655](#6655)) ([5fd7af7](5fd7af7))
* Updated projects-list.json in order to display newly added projects ([#6657](#6657)) ([3a6a103](3a6a103))
* Use correct image name in multi-arch imagetools push step ([faf85e0](faf85e0))
* Use join keys instead of entity names in ODFV materialization ([#6645](#6645)) ([abffebc](abffebc)), closes [#5965](#5965)
* use matching proto class per feature view list in SqliteOnlineStore.plan() ([adb8c1c](adb8c1c)), closes [#6658](#6658)
* Widen Athena integer type mapping for unsigned ints ([3425783](3425783))

### Features

* Add ConnectionRef to DataSource for pluggable external credential resolution ([28bde01](28bde01))
* Add Feature Service Create in UI ([0399380](0399380))
* Add hybrid to ValidOfflineStoreDBStorePersistenceTypes for HybridOfflineStore support ([#6707](#6707)) ([310ab51](310ab51)), closes [#6701](#6701)
* Add MLflow integration support to Feast operator ([#6611](#6611)) ([52999f1](52999f1))
* Add opt-in filter_by_created_timestamp cutoff to get_historical_features ([#6617](#6617)) ([79b33ce](79b33ce)), closes [#6615](#6615)
* Add optional OIDC token audience and issuer verification ([#6670](#6670)) ([ef307c6](ef307c6))
* Add packaged feature repository support to Feast Operator ([8112b1e](8112b1e)), closes [#6598](#6598)
* add plan() support to DynamoDBOnlineStore ([51ce982](51ce982)), closes [#6658](#6658) [#6659](#6659)
* Added optional namespace/colleciton to datasets ([165fcf2](165fcf2))
* Added SQL registry schema_mode and registry create command ([#6704](#6704)) ([037c4cd](037c4cd))
* Allow users to have protected project on shared registry ([f9923bc](f9923bc))
* Apply Intermediate TLS defaults on API fallback and handle transient errors ([#6587](#6587)) ([43ae993](43ae993))
* **cli:** Updated feast init demo by adding rag template ([#5946](#5946)) ([c8628eb](c8628eb)), closes [#5264](#5264)
* Expose the OIDC JWKS tunables through the operator ([#6690](#6690)) ([fef4e78](fef4e78)), closes [#6683](#6683)
* Making feast vector store with open ai search api compatible ([#6121](#6121)) ([54da19a](54da19a))
* Multi-arch publish for feast operator image ([b221036](b221036))
* OpenLineage lineage enhancements - full object coverage, richer UI, and API-level sync ([#6719](#6719)) ([120a868](120a868))
* **operator:** Add spec.services.initImage for init container image override ([#6598](#6598)) ([ca355cb](ca355cb))
* Pass optional OIDC audience and issuer through the operator ([#6677](#6677)) ([a13ed7b](a13ed7b)), closes [#6670](#6670)
* **server:** Remote Materialization ([#6649](#6649)) ([b7ae488](b7ae488)), closes [#4526](#4526)
* Support Lineage configs via operator ([bf1e54a](bf1e54a))
* Updated datasets UI to support grouping ([7ae64ec](7ae64ec))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nquinn408 nquinn408 nquinn408 approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Pre-create feature registry schema

3 participants

@AdityaPatil22 @codecov-commenter @nquinn408