feat: Apply Intermediate TLS defaults on API fallback and handle transient errors by ugiordan · Pull Request #6587 · feast-dev/feast

ugiordan · 2026-07-07T15:55:48Z

What this PR does / why we need it

Follow-up to #6567. Fixes two issues in the TLS profile integration:

Else-only pattern: NewTLSConfigFromProfile was only called in the success branch of the TLS profile fetch. On any error path (non-OpenShift cluster, resource not found), no TLS config was applied, leaving the operator running with Go's bare defaults (no MinVersion, no cipher restrictions). This PR moves NewTLSConfigFromProfile outside the if/else and explicitly falls back to configv1.TLSProfiles[TLSProfileIntermediateType] on all error paths.
No transient error handling: The error switch only handled IsNotFound and IsNoMatchError. Transient API errors (ServiceUnavailable, Timeout, ServerTimeout, TooManyRequests, context.DeadlineExceeded) hit the default case and crashed the operator with os.Exit(1). This PR adds a dedicated case for these errors with a graceful Intermediate fallback. tlsProfileFetched is set to true so the SecurityProfileWatcher self-heals when the API recovers.

Which issue(s) this PR fixes

Follow-up to #6567

Testing

go build ./... in infra/feast-operator
go test ./... in infra/feast-operator

codecov-commenter · 2026-07-16T14:42:14Z

⚠️ Please install the $'codecov app svg image'$ to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 46.40%. Comparing base (3c2ae3c) to head (d993d45).
⚠️ Report is 3 commits behind head on master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

$Impacted file tree graph$

@@            Coverage Diff             @@
##           master    #6587      +/-   ##
==========================================
+ Coverage   46.37%   46.40%   +0.02%     
==========================================
  Files         414      414              
  Lines       50089    50109      +20     
  Branches     7159     7167       +8     
==========================================
+ Hits        23231    23254      +23     
+ Misses      25231    25228       -3     
  Partials     1627     1627

Flag	Coverage Δ
go-feature-server	`30.58% <ø> (ø)`
python-unit	`47.70% <ø> (+0.02%)`	⬆️
see 2 files with indirect coverage changes

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 75b9463...d993d45. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

franciscojavierarceo

The added Notebook ConfigMap controller, validation/polling helpers, RBAC/startup behavior, and os.Exit(0) restart path are unrelated to the TLS fallback fix and substantially enlarge its operational risk. Please split that controller work into a separate PR so the TLS change can be reviewed and reverted independently. The TLS error-classification/default-profile behavior also needs focused unit coverage in this PR; the current diff changes only cmd/main.go and adds no tests.

ugiordan · 2026-07-29T13:06:11Z

The added Notebook ConfigMap controller, validation/polling helpers, RBAC/startup behavior, and os.Exit(0) restart path are unrelated to the TLS fallback fix and substantially enlarge its operational risk. Please split that controller work into a separate PR so the TLS change can be reviewed and reverted independently. The TLS error-classification/default-profile behavior also needs focused unit coverage in this PR; the current diff changes only cmd/main.go and adds no tests.

Thanks for the review @franciscojavierarceo. I've addressed both points:

Removed the Notebook ConfigMap controller code from this PR and will open it as a separate PR.
Extracted the TLS bootstrap logic into cmd/tls_bootstrap.go and added comprehensive unit tests in cmd/tls_bootstrap_test.go covering all error classification edge cases.

The diff is now a single commit touching only TLS error handling.

ntkathole · 2026-07-30T06:14:22Z

+	result.TLSOpts = append(result.TLSOpts, tlsConfigFn)
+
+	adherence, adherenceFetched, err := fetchTLSAdherencePolicy(ctx, k8sClient)
+	if err != nil {


dead code - function returns error but can never return a non-nil error.

ntkathole · 2026-07-30T06:16:12Z

+	result.AdherencePolicy = adherence
+
+	result.TLSOpts = append(result.TLSOpts, func(c *tls.Config) {
+		c.NextProtos = []string{"h2", alpnHTTP11}


alpnHTTP11 is defined as a constant but "h2" is hardcoded inline. Either both should be constants or neither.

ntkathole · 2026-07-30T06:17:49Z

+
+	policy, err := tlspkg.FetchAPIServerTLSAdherencePolicy(fetchCtx, k8sClient)
+	if err != nil {
+		return configv1.TLSAdherencePolicy(""), false, nil


fetchTLSProfile has proper error classification but here silently swallows all errors

Two fixes to the TLS profile integration: 1. NewTLSConfigFromProfile was only called in the success branch of the TLS profile fetch. On error paths (non-OpenShift, not found), no TLS config was applied, leaving Go's bare defaults. Now it always runs with an explicit Intermediate fallback on all error paths. 2. Transient API errors (ServiceUnavailable, Timeout, ServerTimeout, TooManyRequests, DeadlineExceeded) crashed the operator. Now they fall back to Intermediate defaults and set ProfileFetched=true so the SecurityProfileWatcher self-heals when the API recovers. The TLS bootstrap logic is extracted into tls_bootstrap.go with comprehensive unit tests covering all error classification edge cases: transient vs fatal vs graceful fallback, Intermediate defaults always applied, and ALPN configuration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Ugo Giordano <ugiordan@redhat.com>

fetchTLSAdherencePolicy was silently returning nil error on all error paths, making the error check in bootstrapTLS dead code. Apply the same error classification pattern as fetchTLSProfile (NoMatch/NotFound are non-OpenShift fallback, transient errors retry, unexpected errors crash). Also add alpnH2 constant for consistency with alpnHTTP11. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Ugo Giordano <ugiordan@redhat.com>

…sient errors (feast-dev#6587) * fix: handle transient API errors and improve TLS profile fallback Two fixes to the TLS profile integration: 1. NewTLSConfigFromProfile was only called in the success branch of the TLS profile fetch. On error paths (non-OpenShift, not found), no TLS config was applied, leaving Go's bare defaults. Now it always runs with an explicit Intermediate fallback on all error paths. 2. Transient API errors (ServiceUnavailable, Timeout, ServerTimeout, TooManyRequests, DeadlineExceeded) crashed the operator. Now they fall back to Intermediate defaults and set ProfileFetched=true so the SecurityProfileWatcher self-heals when the API recovers. The TLS bootstrap logic is extracted into tls_bootstrap.go with comprehensive unit tests covering all error classification edge cases: transient vs fatal vs graceful fallback, Intermediate defaults always applied, and ALPN configuration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Ugo Giordano <ugiordan@redhat.com> * fix: classify TLS adherence errors instead of swallowing them fetchTLSAdherencePolicy was silently returning nil error on all error paths, making the error check in bootstrapTLS dead code. Apply the same error classification pattern as fetchTLSProfile (NoMatch/NotFound are non-OpenShift fallback, transient errors retry, unexpected errors crash). Also add alpnH2 constant for consistency with alpnHTTP11. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Ugo Giordano <ugiordan@redhat.com> --------- Signed-off-by: Ugo Giordano <ugiordan@redhat.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

# [0.66.0](v0.65.0...v0.66.0) (2026-08-21) ### Bug Fixes * Add connection pre-warming for DynamoDB async client ([89240fa](89240fa)), closes [#6060](#6060) * Add remote registry client extra ([#6697](#6697)) ([b8dfcb0](b8dfcb0)) * Address review feedback on FIPS cipher suite configuration ([4a35fba](4a35fba)) * Allow remote-registry first apply for new projects ([39d408d](39d408d)) * Avoid importing feast.feature_store at mcp_server import time ([ddb2e9a](ddb2e9a)) * Bump pymssql to >=2.3.6 for macOS arm64 wheel support ([181eb35](181eb35)), closes [#5636](#5636) [#5193](#5193) [#5636](#5636) * Call ApplySavedDataset RPC instead of ApplyFeatureService in RemoteRegistry.apply_saved_dataset() ([934d341](934d341)) * Catch missing dbt parser dependency in dbt CLI commands ([#6534](#6534)) ([3c2ae3c](3c2ae3c)) * Default authentication to kubernetes auth ([6a4690a](6a4690a)) * Defer feature-freshness thread to post-fork to avoid Gunicorn deadlock ([#6648](#6648)) ([104ad10](104ad10)), closes [#6647](#6647) * Do not pass undeclared feature view columns to ODFV UDFs ([#6527](#6527)) ([75b9463](75b9463)) * downgrade mcp pin to 1.29.0 and fix CI lockfiles and unit tests ([98e5bca](98e5bca)), closes [#6706](#6706) * Feast apply silently ignoring ttl updates to None or timedelta(0) ([#6709](#6709)) ([97b0f25](97b0f25)), closes [#6703](#6703) * Fix mypy TorchTensor type alias error ([#6712](#6712)) ([34de6fa](34de6fa)), closes [#5563](#5563) * Fixed data source creation form gaps ([5d0f7d6](5d0f7d6)) * Handle parameterized and complex Trino types in type map ([326554d](326554d)) * Isolate default user permissions ([e37adbf](e37adbf)) * Isolate projection join key maps ([d1c709d](d1c709d)) * Map Postgres real to FLOAT instead of DOUBLE ([62db435](62db435)) * Merge shared ODFV source projections in feature resolution ([d269946](d269946)), closes [#6621](#6621) * More exhaustive athena types ([a9aaefc](a9aaefc)) * Normalize SQL registry read_path to the psycopg3 driver like path ([#6644](#6644)) ([996c6ea](996c6ea)), closes [#6643](#6643) * **operator:** add spec.services.onlineStore.disabled to opt out of the online store ([d81d4e3](d81d4e3)), closes [#6586](#6586) * Preinstall DuckDB delta extension for tests ([fd4d49d](fd4d49d)), closes [#6743](#6743) * Preserve event-time ordering within Redis online_write_batch ([40fb788](40fb788)), closes [#5163](#5163) * Prevent mutation of cached feature resolution results ([ea17419](ea17419)) * Remote feastRef FeatureStore fails first apply for a new feastProject ([9affee5](9affee5)) * Remove inert subjectaccessreviews and reorganize RBAC rules ([f771ea4](f771ea4)) * Report single-feature-view spark_application materialization success ([a9219d9](a9219d9)), closes [#6673](#6673) * Reset the global security manager after the permissions fixture ([7667215](7667215)) * Resolve kserve with pip --dry-run instead of installing it ([01da132](01da132)), closes [#6732](#6732) * Resolve write_to_offline_store feature view with a single registry lookup ([a42dc85](a42dc85)), closes [#4235](#4235) * Return False from __eq__ on cross-type comparison ([#6637](#6637)) ([0f149a9](0f149a9)), closes [#6636](#6636) * Reuse IdP-issued client tokens until near expiry ([602d752](602d752)) * Reuse the OIDC JWKS client across requests ([#6683](#6683)) ([a1e6fc2](a1e6fc2)) * Separate CronJob and feature-server ServiceAccounts ([398f643](398f643)) * Serialize UnixTimestamp proto values as raw int64 in remote online store transport ([1e7134f](1e7134f)) * Set FIPS cipher suites before pyarrow.flight import to prevent crash on IBM Power ([979b82a](979b82a)) * Support Entra ID (Azure AD) token claims in OIDC auth ([#6631](#6631)) ([f843c63](f843c63)) * UDF/ODFV source rehydrate (+ Postgres / online cache) ([#6655](#6655)) ([5fd7af7](5fd7af7)) * Updated projects-list.json in order to display newly added projects ([#6657](#6657)) ([3a6a103](3a6a103)) * Use correct image name in multi-arch imagetools push step ([faf85e0](faf85e0)) * Use join keys instead of entity names in ODFV materialization ([#6645](#6645)) ([abffebc](abffebc)), closes [#5965](#5965) * use matching proto class per feature view list in SqliteOnlineStore.plan() ([adb8c1c](adb8c1c)), closes [#6658](#6658) * Widen Athena integer type mapping for unsigned ints ([3425783](3425783)) ### Features * Add ConnectionRef to DataSource for pluggable external credential resolution ([28bde01](28bde01)) * Add Feature Service Create in UI ([0399380](0399380)) * Add hybrid to ValidOfflineStoreDBStorePersistenceTypes for HybridOfflineStore support ([#6707](#6707)) ([310ab51](310ab51)), closes [#6701](#6701) * Add MLflow integration support to Feast operator ([#6611](#6611)) ([52999f1](52999f1)) * Add opt-in filter_by_created_timestamp cutoff to get_historical_features ([#6617](#6617)) ([79b33ce](79b33ce)), closes [#6615](#6615) * Add optional OIDC token audience and issuer verification ([#6670](#6670)) ([ef307c6](ef307c6)) * Add packaged feature repository support to Feast Operator ([8112b1e](8112b1e)), closes [#6598](#6598) * add plan() support to DynamoDBOnlineStore ([51ce982](51ce982)), closes [#6658](#6658) [#6659](#6659) * Added optional namespace/colleciton to datasets ([165fcf2](165fcf2)) * Added SQL registry schema_mode and registry create command ([#6704](#6704)) ([037c4cd](037c4cd)) * Allow users to have protected project on shared registry ([f9923bc](f9923bc)) * Apply Intermediate TLS defaults on API fallback and handle transient errors ([#6587](#6587)) ([43ae993](43ae993)) * **cli:** Updated feast init demo by adding rag template ([#5946](#5946)) ([c8628eb](c8628eb)), closes [#5264](#5264) * Expose the OIDC JWKS tunables through the operator ([#6690](#6690)) ([fef4e78](fef4e78)), closes [#6683](#6683) * Making feast vector store with open ai search api compatible ([#6121](#6121)) ([54da19a](54da19a)) * Multi-arch publish for feast operator image ([b221036](b221036)) * OpenLineage lineage enhancements - full object coverage, richer UI, and API-level sync ([#6719](#6719)) ([120a868](120a868)) * **operator:** Add spec.services.initImage for init container image override ([#6598](#6598)) ([ca355cb](ca355cb)) * Pass optional OIDC audience and issuer through the operator ([#6677](#6677)) ([a13ed7b](a13ed7b)), closes [#6670](#6670) * **server:** Remote Materialization ([#6649](#6649)) ([b7ae488](b7ae488)), closes [#4526](#4526) * Support Lineage configs via operator ([bf1e54a](bf1e54a)) * Updated datasets UI to support grouping ([7ae64ec](7ae64ec))

$@ugiordan$

ugiordan requested a review from a team as a code owner July 7, 2026 15:55

$@ugiordan$

ugiordan force-pushed the feat/tls-intermediate-fallback branch 6 times, most recently from 1375b6f to 748ecc5 Compare July 9, 2026 08:38

$@ntkathole$

ntkathole force-pushed the feat/tls-intermediate-fallback branch from 748ecc5 to fe29daa Compare July 14, 2026 06:01

$@ntkathole$ ntkathole added the ok-to-test label Jul 14, 2026

$@ugiordan$

ugiordan force-pushed the feat/tls-intermediate-fallback branch 2 times, most recently from da15060 to c7b9f67 Compare July 17, 2026 13:33

$@ugiordan$

ugiordan force-pushed the feat/tls-intermediate-fallback branch from 1a7c340 to 4636f78 Compare July 24, 2026 12:01

$@ugiordan$ ugiordan mentioned this pull request Jul 24, 2026

fix: handle transient errors and IsServerTimeout in TLS profile resolution opendatahub-io/feast#160

Closed

$@ugiordan$

ugiordan force-pushed the feat/tls-intermediate-fallback branch from 4636f78 to 7f2abf8 Compare July 29, 2026 11:43

$franciscojavierarceo$

franciscojavierarceo requested changes Jul 29, 2026

View reviewed changes

$@ugiordan$

ugiordan force-pushed the feat/tls-intermediate-fallback branch 2 times, most recently from 228829a to ef81603 Compare July 29, 2026 13:02

$@ugiordan$

ugiordan force-pushed the feat/tls-intermediate-fallback branch 2 times, most recently from 9083270 to bd45c31 Compare July 29, 2026 13:45

$ntkathole$

ntkathole reviewed Jul 30, 2026

View reviewed changes

$@ugiordan$

ugiordan force-pushed the feat/tls-intermediate-fallback branch 3 times, most recently from edde11a to a7823be Compare July 30, 2026 08:42

ugiordan and others added 2 commits July 30, 2026 16:41

$@ntkathole$

ntkathole force-pushed the feat/tls-intermediate-fallback branch from a7823be to d993d45 Compare July 30, 2026 11:11

$ntkathole$

ntkathole approved these changes Jul 30, 2026

View reviewed changes

$@ntkathole$

ntkathole merged commit 43ae993 into feast-dev:master Jul 30, 2026
19 of 23 checks passed

$@ugiordan$ ugiordan mentioned this pull request Aug 10, 2026

fix: classify TLS adherence errors instead of silently swallowing them opendatahub-io/opendatahub-operator#3947

Merged

8 tasks

$@ntkathole$ ntkathole mentioned this pull request Aug 21, 2026

fix: Copy entire cmd/ directory in operator Konflux Dockerfile red-hat-data-services/feast#335

Merged

2 tasks

Navigation Menu

Uh oh!

feat: Apply Intermediate TLS defaults on API fallback and handle transient errors - #6587

feat: Apply Intermediate TLS defaults on API fallback and handle transient errors#6587
ntkathole merged 2 commits into
feast-dev:masterfrom
ugiordan:feat/tls-intermediate-fallback

$@ugiordan$ ugiordan commented Jul 7, 2026 •

edited

Loading

Uh oh!

codecov-commenter commented Jul 16, 2026 •

edited

Loading

Uh oh!

$@franciscojavierarceo$ franciscojavierarceo left a comment

Uh oh!

ugiordan commented Jul 29, 2026

Uh oh!

$@ntkathole$ ntkathole Jul 30, 2026

Uh oh!

$@ntkathole$ ntkathole Jul 30, 2026

Uh oh!

$@ntkathole$ ntkathole Jul 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

ugiordan commented Jul 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it

Which issue(s) this PR fixes

Testing

Uh oh!

codecov-commenter commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

franciscojavierarceo left a comment

Choose a reason for hiding this comment

Uh oh!

ugiordan commented Jul 29, 2026

Uh oh!

ntkathole Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

ntkathole Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

ntkathole Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

$@ugiordan$ ugiordan commented Jul 7, 2026 •

edited

Loading

codecov-commenter commented Jul 16, 2026 •

edited

Loading

$@franciscojavierarceo$ franciscojavierarceo left a comment

$@ntkathole$ ntkathole Jul 30, 2026

$@ntkathole$ ntkathole Jul 30, 2026

$@ntkathole$ ntkathole Jul 30, 2026