Identity provider tokens

When Gravitino runs with OAuth authentication, every request from an engine carries a bearer token issued by your identity provider (IdP). Spark, Trino, PyIceberg, and the other engines all accept that token as a catalog property, but none of them tells you how to obtain it. The scripts on this page do that for Microsoft Entra ID, Okta, Keycloak, and Auth0, for both a person signing in and a workload running unattended.

The scripts produce a token. Gravitino decides whether to accept it, using the server settings described in Token requirements. Most failed logins come from a token that is valid for the IdP but carries the wrong audience or issuer for Gravitino, so check a new token with the decode script before handing it to an engine.

Quick start

The fastest path is the client credentials grant, which needs no browser. Every provider on this page supports it with the same request shape; only the endpoint and scope differ.

1. Request a token. Set the three variables from your IdP's app registration, then call the token endpoint.

   get-token.sh

   TOKEN_ENDPOINT="{token_endpoint}"
   CLIENT_ID="{client_id}"
   CLIENT_SECRET="{client_secret}"
   SCOPE="{scope}"
   
   TOKEN=$(curl -s -X POST "$TOKEN_ENDPOINT" \
     -d grant_type=client_credentials \
     -d client_id="$CLIENT_ID" \
     -d client_secret="$CLIENT_SECRET" \
     -d scope="$SCOPE" | jq -r .access_token)
   
   echo "$TOKEN"

2. Check the claims. Decode the payload and compare iss, aud, and the principal claim against the Gravitino settings for your provider in Token requirements.

   decode-token.sh

   python3 -c 'import sys,json,base64; p=sys.argv[1].split(".")[1]; print(json.dumps(json.loads(base64.urlsafe_b64decode(p+"="*(-len(p)%4))),indent=2))' "$TOKEN"

3. Give the token to the engine. For example, in PyIceberg:

   query.py

   from pyiceberg.catalog import load_catalog
   
   catalog = load_catalog(
       "gravitino",
       type="rest",
       uri="https://{iceberg_host}/iceberg",
       token="{token}",
   )
   print(catalog.list_namespaces())

The per-provider sections below give the exact endpoints and scopes, plus browser and device code scripts for people signing in as themselves.

Token requirements

Gravitino validates tokens with the JWKS validator. It accepts a token only when all of the following hold:

• The signature verifies against a key published at gravitino.authenticator.oauth.jwksUri.
• The iss claim matches gravitino.authenticator.oauth.authority exactly, including any trailing slash.
• The aud claim contains gravitino.authenticator.oauth.serviceAudience.
• The token has not expired, allowing for gravitino.authenticator.oauth.allowSkewSecs.
• One of the claims listed in gravitino.authenticator.oauth.principalFields is present. Gravitino takes the first one it finds as the user name. The default is sub.

Group membership comes from the claims listed in gravitino.authenticator.oauth.groupsFields, which defaults to groups. A token without a groups claim is still accepted; the user simply arrives with no IdP groups.

my-values.yaml

gravitino:
  authenticators: "oauth,basic"
  authenticator:
    oauth:
      tokenValidatorClass: "org.apache.gravitino.server.authentication.JwksTokenValidator"
      jwksUri: "{jwks_uri}"
      authority: "{issuer}"
      serviceAudience: "{audience}"
      principalFields: "{principal_claim}"
      groupsFields: "{groups_claim}"

Apply the values by rerunning the install command from Install. The Authentication page covers each setting in full.

The values for each provider:

Provider	authority (token iss)	jwksUri	serviceAudience (token aud)	principalFields
Entra ID	https://login.microsoftonline.com/{tenant_id}/v2.0	https://login.microsoftonline.com/{tenant_id}/discovery/v2.0/keys	{gravitino_app_id}	preferred_username,oid
Okta	https://{okta_domain}/oauth2/default	https://{okta_domain}/oauth2/default/v1/keys	api://default	sub
Keycloak	https://{keycloak_host}/realms/{realm}	https://{keycloak_host}/realms/{realm}/protocol/openid-connect/certs	gravitino	preferred_username
Auth0	https://{auth0_domain}/	https://{auth0_domain}/.well-known/jwks.json	{api_identifier}	sub

Each provider's section explains where these values come from and the setting most likely to produce a token Gravitino rejects.

Grant types

Three OAuth grants cover nearly every case.

• Authorization code with PKCE. A person signs in through a browser on the machine running the script. Use it for notebooks and desktop tools.
• Device code. The script prints a short code and a URL, and the person signs in on any browser, including a phone. Use it on a remote box, in an SSH session, or anywhere a local browser cannot receive a redirect.
• Client credentials. No person is involved; the workload authenticates with its own client ID and secret. Use it for scheduled jobs, pipelines, and services. The resulting principal is the service identity, so grant that identity its own roles in Gravitino.

User tokens typically expire after an hour. A pasted token stops working when it expires, so a long-running job should use client credentials and let the engine fetch tokens itself, as described in Engine configuration.

Provider scripts

The browser and device code scripts are written in Python. The Entra ID scripts use the Microsoft Authentication Library (pip install msal); the others use requests against the provider's standard OAuth endpoints, so the same device code script serves Okta, Keycloak, and Auth0.

Microsoft Entra ID

Entra ID needs two app registrations: one representing Gravitino as an API, and one for each client that requests tokens.

1. Register the Gravitino API. Create an app registration named for Gravitino. Under Expose an API, set the Application ID URI to api://{gravitino_app_id} and add a scope such as catalog.access. In the app manifest, set the access token version to 2 (requestedAccessTokenVersion: 2, shown as accessTokenAcceptedVersion in the older manifest format).
2. Register the client. For people signing in, create a public client registration, add the redirect URI http://localhost under Mobile and desktop applications, enable Allow public client flows for device code, and grant it the delegated catalog.access permission. For workloads, create a confidential client with a client secret and grant it an application permission (an app role) on the Gravitino API.
3. Emit groups if you use them. Add a groups claim under Token configuration. Entra ID emits group object IDs, not names, so map them in Gravitino with a group mapper or create Gravitino groups named by object ID.

The access token version is the setting most often missed. With version 1, Entra ID issues tokens with iss set to https://sts.windows.net/{tenant_id}/ and aud set to api://{gravitino_app_id}, so neither matches the values in the table above. Either switch the API to version 2 or configure Gravitino with the version 1 values. Requesting a Microsoft Graph scope such as User.Read instead of the Gravitino scope produces a token for Graph, which no third party can validate.

Browser

entra_browser.py

import msal

TENANT_ID = "{tenant_id}"
CLIENT_ID = "{client_id}"
SCOPE = "api://{gravitino_app_id}/catalog.access"

app = msal.PublicClientApplication(
    CLIENT_ID,
    authority=f"https://login.microsoftonline.com/{TENANT_ID}",
)

# Opens the system browser and listens on localhost for the redirect.
# MSAL adds PKCE automatically.
result = app.acquire_token_interactive(scopes=[SCOPE])

if "access_token" not in result:
    raise SystemExit(f"{result.get('error')}: {result.get('error_description')}")
print(result["access_token"])

Device code

entra_device.py

import msal

TENANT_ID = "{tenant_id}"
CLIENT_ID = "{client_id}"
SCOPE = "api://{gravitino_app_id}/catalog.access"

app = msal.PublicClientApplication(
    CLIENT_ID,
    authority=f"https://login.microsoftonline.com/{TENANT_ID}",
)

flow = app.initiate_device_flow(scopes=[SCOPE])
if "user_code" not in flow:
    raise SystemExit(f"Device flow failed: {flow}")
print(flow["message"])  # Tells the user which URL to open and which code to enter.

result = app.acquire_token_by_device_flow(flow)  # Blocks until sign-in completes.

if "access_token" not in result:
    raise SystemExit(f"{result.get('error')}: {result.get('error_description')}")
print(result["access_token"])

Client credentials

entra_client.py

import msal

TENANT_ID = "{tenant_id}"
CLIENT_ID = "{client_id}"
CLIENT_SECRET = "{client_secret}"
# Client credentials always request the .default scope of the target API.
SCOPE = "api://{gravitino_app_id}/.default"

app = msal.ConfidentialClientApplication(
    CLIENT_ID,
    authority=f"https://login.microsoftonline.com/{TENANT_ID}",
    client_credential=CLIENT_SECRET,
)

result = app.acquire_token_for_client(scopes=[SCOPE])

if "access_token" not in result:
    raise SystemExit(f"{result.get('error')}: {result.get('error_description')}")
print(result["access_token"])

The token endpoint for engines that fetch their own tokens is https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token. A client credentials token has no preferred_username; Gravitino falls through to oid, the object ID of the client's service principal.

Okta

Use a custom authorization server. Tokens from the Okta org authorization server (https://{okta_domain}/oauth2/v1/..., with no server ID in the path) are intended for Okta's own APIs and cannot be validated by Gravitino. The examples use the server named default; substitute your own server ID if you created one.

1. Define a scope. On the authorization server, add a scope such as catalog.access. Client credentials requests must name a custom scope, because openid is not allowed in that grant.
2. Create the client. For people, create an OIDC native application and enable the Device Authorization grant, the Authorization Code grant, or both. For workloads, create an API services application, which uses client credentials.
3. Allow the grant in an access policy. The authorization server's access policy must include a rule permitting the grant type and the client, or Okta rejects the request.
4. Emit groups if you use them. Okta access tokens carry no groups claim by default. Add a claim named groups to the authorization server, included in access tokens, with a group filter.

The audience is set on the authorization server and defaults to api://default. A user token's sub is the user's Okta login; a client credentials token's sub is the client ID.

Endpoint	URL
Token	https://{okta_domain}/oauth2/default/v1/token
Device code	https://{okta_domain}/oauth2/default/v1/device/authorize
Authorize	https://{okta_domain}/oauth2/default/v1/authorize

okta-client.sh

OKTA_DOMAIN="{okta_domain}"
CLIENT_ID="{client_id}"
CLIENT_SECRET="{client_secret}"

curl -s -X POST "https://$OKTA_DOMAIN/oauth2/default/v1/token" \
  -u "$CLIENT_ID:$CLIENT_SECRET" \
  -d grant_type=client_credentials \
  -d scope=catalog.access | jq -r .access_token

For people signing in, run the device code script with the device code and token endpoints above and the scope openid catalog.access.

Keycloak

1. Give Gravitino its own audience. Keycloak access tokens carry aud: account by default, and the Keycloak example on the Authentication page accepts that value. Every Keycloak client in the realm can obtain an account token, so a dedicated audience is the better production choice. Create a client scope with an Audience mapper whose included custom audience is gravitino, set it to add to the access token, assign the client scope as a default scope to every client that calls Gravitino, and set serviceAudience to gravitino.
2. Create the client. For people, enable OAuth 2.0 Device Authorization Grant, Standard flow, or both on the client. For workloads, create a confidential client with Client authentication and Service accounts roles enabled.
3. Emit groups if you use them. Add a Group Membership mapper with the token claim name groups. With Full group path on, group names arrive as /analysts; either turn it off or strip the leading slash with the Gravitino group mapper pattern ^/(.*) described in Authentication.

A user token's preferred_username is the Keycloak user name. A client credentials token's preferred_username is service-account-{client_id}.

Endpoint	URL
Token	https://{keycloak_host}/realms/{realm}/protocol/openid-connect/token
Device code	https://{keycloak_host}/realms/{realm}/protocol/openid-connect/auth/device
Authorize	https://{keycloak_host}/realms/{realm}/protocol/openid-connect/auth

keycloak-client.sh

KEYCLOAK_HOST="{keycloak_host}"
REALM="{realm}"
CLIENT_ID="{client_id}"
CLIENT_SECRET="{client_secret}"

curl -s -X POST "https://$KEYCLOAK_HOST/realms/$REALM/protocol/openid-connect/token" \
  -d grant_type=client_credentials \
  -d client_id="$CLIENT_ID" \
  -d client_secret="$CLIENT_SECRET" | jq -r .access_token

For people signing in, run the device code script with the device code and token endpoints above and the scope openid.

Auth0

1. Register Gravitino as an API. Create an API in Auth0 and give it an identifier such as https://gravitino.example.com. The identifier becomes the token's aud and the Gravitino serviceAudience.
2. Create the client. For people, create a native application and enable the Device Code grant under its advanced settings. For workloads, create a machine-to-machine application and authorize it for the Gravitino API.
3. Emit groups if you use them. Auth0 has no built-in groups claim. Add one with a post-login Action under a namespaced name such as https://gravitino.example.com/groups, and set groupsFields to that name.

Every token request must include the audience parameter. Without it, Auth0 issues a token for its own user info endpoint, which Gravitino cannot validate. The issuer ends with a slash, and authority must include it. A user token's sub looks like auth0|64f0...; a client credentials token's sub is {client_id}@clients. If you want email addresses as Gravitino user names, add an email claim in the same Action and list it first in principalFields.

Endpoint	URL
Token	https://{auth0_domain}/oauth/token
Device code	https://{auth0_domain}/oauth/device/code
Authorize	https://{auth0_domain}/authorize

auth0-client.sh

AUTH0_DOMAIN="{auth0_domain}"
CLIENT_ID="{client_id}"
CLIENT_SECRET="{client_secret}"
AUDIENCE="{api_identifier}"

curl -s -X POST "https://$AUTH0_DOMAIN/oauth/token" \
  -d grant_type=client_credentials \
  -d client_id="$CLIENT_ID" \
  -d client_secret="$CLIENT_SECRET" \
  -d audience="$AUDIENCE" | jq -r .access_token

For people signing in, run the device code script with the endpoints above, the scope openid, and AUDIENCE set to the API identifier.

Device code script

The script implements the standard device authorization grant (RFC 8628) and works with any provider that supports it. Set the endpoints and scope from the provider's section; AUDIENCE is needed only for Auth0.

device_token.py

import time
import requests

DEVICE_ENDPOINT = "{device_code_endpoint}"
TOKEN_ENDPOINT = "{token_endpoint}"
CLIENT_ID = "{client_id}"
SCOPE = "{scope}"
AUDIENCE = None  # Auth0 only: the API identifier.

params = {"client_id": CLIENT_ID, "scope": SCOPE}
if AUDIENCE:
    params["audience"] = AUDIENCE

device = requests.post(DEVICE_ENDPOINT, data=params).json()
if "device_code" not in device:
    raise SystemExit(f"Device authorization failed: {device}")

url = device.get("verification_uri_complete") or device["verification_uri"]
print(f"Open {url} and enter the code {device['user_code']}")

interval = device.get("interval", 5)
deadline = time.time() + device.get("expires_in", 600)

while time.time() < deadline:
    time.sleep(interval)
    resp = requests.post(TOKEN_ENDPOINT, data={
        "grant_type": "urn:ietf:params:oauth:grant-type:device_code",
        "device_code": device["device_code"],
        "client_id": CLIENT_ID,
    }).json()

    if "access_token" in resp:
        print(resp["access_token"])
        break
    error = resp.get("error")
    if error == "authorization_pending":
        continue
    if error == "slow_down":
        interval += 5
        continue
    raise SystemExit(f"{error}: {resp.get('error_description')}")
else:
    raise SystemExit("The device code expired before sign-in completed.")

Token inspection

Before giving a token to an engine, decode it and compare the claims with the Gravitino settings. A token rejected with a 401 almost always shows the mismatch here. The script only decodes the payload; it does not verify the signature, and it is safe to run on any token you hold.

decode_token.py

import base64
import json
import sys
import time

token = sys.argv[1]
payload = token.split(".")[1]
payload += "=" * (-len(payload) % 4)
claims = json.loads(base64.urlsafe_b64decode(payload))

print(json.dumps(claims, indent=2))
print()
print("iss:", claims.get("iss"))
print("aud:", claims.get("aud"))
print("expires in:", int(claims.get("exp", 0) - time.time()), "seconds")

Run the inspection

python decode_token.py "$TOKEN"

A token that is not three dot-separated segments is not a JWT. Auth0 returns one when the audience parameter is missing, and Gravitino cannot validate it.

Engine configuration

Each engine takes a ready-made token in one property. Use it for interactive work with a token from the browser or device code scripts.

Engine	Static token property	Client credentials properties
Spark	spark.sql.catalog.{catalog_name}.token	spark.sql.catalog.{catalog_name}.credential, .oauth2-server-uri, .scope
Trino	iceberg.rest-catalog.oauth2.token	iceberg.rest-catalog.oauth2.credential, iceberg.rest-catalog.oauth2.server-uri, iceberg.rest-catalog.oauth2.scope
PyIceberg	token	credential, oauth2-server-uri, scope

The credential property takes {client_id}:{client_secret}, and oauth2-server-uri takes the provider's token endpoint. With these set, the engine runs the client credentials grant itself and requests a new token before the old one expires, so a long-running job never fails on an expired token. For Auth0, the Iceberg client has no property for the audience parameter; set the API as the default audience for the tenant, or fetch the token with the script above and pass it as a static token.

Spark with a static token

spark-sql \
  --conf spark.sql.catalog.gravitino=org.apache.iceberg.spark.SparkCatalog \
  --conf spark.sql.catalog.gravitino.type=rest \
  --conf spark.sql.catalog.gravitino.uri=https://{iceberg_host}/iceberg \
  --conf spark.sql.catalog.gravitino.token="$TOKEN"

etc/catalog/gravitino.properties

connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=https://{iceberg_host}/iceberg
iceberg.rest-catalog.security=OAUTH2
iceberg.rest-catalog.oauth2.credential={client_id}:{client_secret}
iceberg.rest-catalog.oauth2.server-uri={token_endpoint}
iceberg.rest-catalog.oauth2.scope={scope}

A token is a credential. Pass it through an environment variable or a secret store rather than writing it into a catalog file or a notebook that others can read.

Related

• Authentication
• Client access
• Credential vending
• Iceberg REST catalog service
• Trino
• Spark
• PyIceberg