Skip to content

The use of JWT in Trino offline store failes #6760

Description

@kow-butterfly

Expected Behavior

The call of def run_get_historical below should be able to reach trino.

Current Behavior

Unable to use a jwt token to authenticate against trino.

Steps to reproduce

Offline store like:

offline_store:
    type: trino
...
    user: xyz
    # Enables authentication in Trino connections, pick the one you need
    auth:
        type: jwt
        config:
            token: ${TRINO_JWT_TOKEN}

Setting the variable like with an inhouse keycloak instance:

export TRINO_JWT_TOKEN=$(curl --location 'https://foo.bar/' \
  --header 'Content-Type: application/x-www-form-urlencoded' \
  --data-urlencode 'username=<USER_NAME>' \
  --data-urlencode 'password=<PASSWORD>' | jq -r '.access_token')

Running a historical query

def run_get_historical(a_date):
    import pandas as pd
    entity_df = pd.DataFrame(
        {
            "schadennr": ["123", "456"],
            "event_timestamp": [a_date, a_date],
        }
    )
    feature_df = store.get_historical_features(
        entity_df=entity_df,
        features=store.get_feature_service("train_service")
    ).to_df()
    print(feature_df.head(3).drop(columns=["schadennr"]))

    # a_date: Optional[Union[str, datetime]] = None

def main() -> int:
    an_optional_date_string = sys.argv[1] if len(sys.argv) > 1 else None
    if an_optional_date_string is None:
        a_date = datetime.now()
        print(a_date)
    elif isinstance(an_optional_date_string, str):
        a_date = datetime.strptime(an_optional_date_string, "%Y-%m-%d")
        print(a_date)
    else:
        raise ValueError("Invalid date format. Please provide a date string in 'YYYY-MM-DD' format.")
    run_get_historical(a_date)
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

Fails with:

$ python run_feast_offline_query.py
2026-08-19 16:01:27.655863
Traceback (most recent call last):
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/trino_project/feature_repo/run_feast_offline_query.py", line 48, in <module>
    raise SystemExit(main())
                     ^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/trino_project/feature_repo/run_feast_offline_query.py", line 44, in main
    run_get_historical(a_date)
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/trino_project/feature_repo/run_feast_offline_query.py", line 26, in run_get_historical
    feature_df = store.get_historical_features(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/feature_store.py", line 2025, in get_historical_features
    job = provider.get_historical_features(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/infra/passthrough_provider.py", line 481, in get_historical_features
    job = self.offline_store.get_historical_features(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/infra/offline_stores/contrib/trino_offline_store/trino.py", line 371, in get_historical_features
    entity_schema = _upload_entity_df_and_get_entity_schema(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/infra/offline_stores/contrib/trino_offline_store/trino.py", line 515, in _upload_entity_df_and_get_entity_schema
    upload_pandas_dataframe_to_trino(
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/infra/offline_stores/contrib/trino_offline_store/connectors/upload.py", line 180, in upload_pandas_dataframe_to_trino
    client.execute_query(
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/infra/offline_stores/contrib/trino_offline_store/trino_queries.py", line 87, in execute_query
    return query.execute()
           ^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/feast/infra/offline_stores/contrib/trino_offline_store/trino_queries.py", line 105, in execute
    self._cursor.execute(operation=self.query_text)
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/trino/dbapi.py", line 642, in execute
    self._iterator = iter(self._query.execute())
                          ^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/trino/client.py", line 952, in execute
    response = self._request.post(self._query, additional_http_headers)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/trino/client.py", line 681, in post
    http_response = self._post(
                    ^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/trino/client.py", line 1107, in decorated
    raise error
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/trino/client.py", line 1089, in decorated
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/requests/sessions.py", line 712, in post
    return self.request("POST", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/requests/sessions.py", line 635, in request
    prep = self.prepare_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/requests/sessions.py", line 541, in prepare_request
    p.prepare(
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/requests/models.py", line 443, in prepare
    self.prepare_auth(auth, url)
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/requests/models.py", line 689, in prepare_auth
    r = auth_handler(self)
        ^^^^^^^^^^^^^^^^^^
  File "/home/xyz/bitbucket-repos/dat/xyz_feast/.venv/lib/python3.11/site-packages/trino/auth.py", line 257, in __call__
    r.headers["Authorization"] = "Bearer " + self.token
                                 ~~~~~~~~~~^~~~~~~~~~~~
TypeError: can only concatenate str (not "SecretStr") to str

Specifications

  • Version: feast 0.65.0
  • Platform: RHEL 9.8 (Plow), 5.14.0-687.17.1.el9_8.x86_64
  • Subsystem:

Possible Solution

When patching the code like:

_original_init = trino.auth.JWTAuthentication.__init__


def _patched_init(self, token):
    if hasattr(token, "get_secret_value"):
        token = token.get_secret_value()
    _original_init(self, str(token))


trino.auth.JWTAuthentication.__init__ = _patched_init

before our code runs, ensures that the call to trino succeeds.

Somehow the token gets packed into a SecretStr but when the python code wants to use the token, it expects it to be a simple string.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions