fix: Require signed tokens and a private secret for intra-server communication - #6801
fix: Require signed tokens and a private secret for intra-server communication#6801hiyufan wants to merge 2 commits into
Conversation
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6801 +/- ##
==========================================
+ Coverage 47.08% 47.12% +0.03%
==========================================
Files 419 420 +1
Lines 51878 51912 +34
Branches 7525 7529 +4
==========================================
+ Hits 24429 24463 +34
- Misses 25700 25701 +1
+ Partials 1749 1748 -1
... and 1 file with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
…unication
The RBAC layer granted a fully trusted intra-communication identity to any
request whose `preferred_username` (OIDC) or service-account name (Kubernetes)
claim matched `INTRA_COMMUNICATION_BASE64`. That claim was read from an
unverified JWT decode performed before any signature check, and the value it
was compared against is hardcoded in Feast's own Helm chart to the constant
`base64("intra-server-communication")`, so any caller able to reach a Feast
server could mint an unsigned token and skip every permission check on every
project that server serves.
Sign the intra-server token with HS256 using the shared secret and verify that
signature before reading any claim, so a caller without the secret cannot mint
an acceptable token. Take the secret from a Kubernetes Secret named by the new
`intraCommunicationSecret` chart value and omit the variable when it is not
configured, since signing does not help while the key is a constant published
in this repository. Treat an unset or empty secret as "disabled" rather than a
value to compare against.
This is a breaking change: deployments that relied on the chart default must
create a Secret and point every Feast release that talks to another one at it.
Fixes feast-dev#6785
Signed-off-by: Chen Yufan <yufan_ai@outlook.com>
93a26b2 to
488a5e4
Compare
|
Pushed a fix for the lint failure: The I did check whether the new import in
A 2.5s import against a 60s timeout, so I read that job as runner contention. Happy to dig further if it reproduces. |
The Kubernetes parser reads the service-account name out of a token whose signature already verified, and rejects it when the subject is absent, is not a string, does not split into four segments, or names a different account. None of those branches had a test: the existing cases stop at the signature check, so the subject handling below it was reached only on the happy path. Add the rejected subjects, and the matching case for the OIDC parser's `preferred_username`, so a holder of the shared secret cannot reach the internal identity with a token whose subject does not name it. Signed-off-by: Chen Yufan <yufan_ai@outlook.com>
|
Pushed a test-only commit for the patch-coverage gap Codecov flagged in The uncovered lines were the subject handling in Behaviour of the helper with the secret configured, one line per branch:
One note on the earlier Codecov comment: it was measured against I could not produce a coverage percentage locally — |
What this PR does / why we need it:
Feast's RBAC layer grants a fully trusted intra-communication identity to any request whose
preferred_username(OIDC) or service-account name (Kubernetes) claim matchesINTRA_COMMUNICATION_BASE64. That claim was read from an unverified JWT decode performedbefore any signature check or IdP round-trip, and the value it was compared against is hardcoded
in Feast's own Helm chart to the constant
base64("intra-server-communication").Any caller able to reach a Feast server could therefore mint
and have
is_auth_necessary()skip every permission check, for every project on that server. Theadvisory's PoC reads all permission policies, reads across projects, and creates and deletes
entities, using a token with an empty signature and no roles.
The fix
Both halves have to change, because either one on its own leaves the identity forgeable:
IntraCommAuthClientManagersigns withHS256using the sharedsecret instead of
algorithm="none", and both token parsers verify that signature beforereading any claim. A caller that does not hold the secret cannot produce an acceptable token,
whatever it writes in the payload.
INTRA_COMMUNICATION_BASE64now comes from aKubernetes Secret named by the new
intraCommunicationSecretvalue, and the variable isomitted entirely when that is not configured. Signing alone would not help while the key is a
constant published in this repository.
get_intra_comm_secret()normalizes""toNone, andis_auth_necessary()now requires aconfigured secret before it will skip permission checks, so a blank username cannot match a
blank environment variable.
The shared logic lives in a new
feast/permissions/auth/intra_comm.py, so the OIDC parser, theKubernetes parser and the client agree on how the token is minted and verified.
Deployments that relied on the chart's default must now create a Secret and set
intraCommunicationSecret.name. Every Feast release that calls another one (a feature server andthe registry server it reads from, for example) has to point at the same Secret, and the value has
to be private to the installation. Until it is configured, intra-server communication is disabled
rather than silently trusting a well-known value: that fail-closed default is deliberate, since
the alternative is the vulnerability being fixed.
Which issue(s) this PR fixes:
Fixes #6785
Advisory: GHSA-h543-6vgr-fm36
Checks
git commit -s)Testing Strategy
Behaviour, measured before and after
alg=none,preferred_username= the chart's constantHS256signed with a different secretHS256signed with the configured secretUnit tests
test_oidc_inter_server_commandtest_k8s_inter_server_commpreviously drove the intra-serverpath with a placeholder token and a monkeypatched
jwt.decode. Once the signature is checked theyno longer reach that path at all, so they now build a genuinely signed token and assert on it.
New cases cover what the old ones could not:
test_oidc_intra_comm_rejects_forged_token/test_k8s_intra_comm_rejects_forged_token—unsigned, and signed with another secret
test_intra_comm_disabled_without_secret— unset and emptytest_intra_comm_token_verifies_only_with_its_own_secrettest_intra_comm_client_token_is_accepted_by_oidc_parser— the client and the parser stillagree end to end
Helm chart
ruff check,ruff format --checkandmypypass on every changed file.Misc
Two design points worth a reviewer's opinion:
which keeps the change small and leaves
is_auth_necessary()semantics untouched, but it meansan observer of one internal request learns the secret. Moving the claim to a fixed non-secret
marker and relying on the signature alone would fix that; it needs a different
is_auth_necessary()check, so I left it out of a security fix that should stay reviewable.Happy to add it here or follow up separately.
mixed-version rolling upgrade will see intra-server calls fail until both sides are updated. If
you would rather have a release that accepts both forms first, say so and I will add a
transition path.