Iceberg REST Catalog Service
Overview
Gravitino serves an Iceberg REST catalog (IRC) that implements the Apache Iceberg REST catalog specification. Any engine with an Iceberg REST client, including Spark, Trino, Flink, and PyIceberg, reads and writes Iceberg tables through it with no Gravitino connector installed.
The IRC serves the Iceberg catalogs of a metalake. A catalog created in Gravitino with the
lakehouse-iceberg provider is available through the IRC as soon as it exists, under its own name,
and every request is authenticated and authorized by Gravitino against the same privileges that
govern the Gravitino API.
| Item | Iceberg REST Catalog | Gravitino API |
|---|---|---|
| Protocol | The Apache Iceberg REST specification | The Gravitino REST API |
| Serves | Iceberg tables, namespaces, and views | Every catalog type Gravitino connects |
| Used by | Engines with an Iceberg REST client | The UI, Gravitino connectors, and client libraries |
Quick Start
1. Create an Iceberg catalog in the metalake. See Iceberg Catalog for the backend and storage properties.
2. Point an engine at the IRC. The catalog name is the warehouse. This connects Spark with a bearer token and asks for vended storage credentials:
./bin/spark-sql \
--conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions \
--conf spark.sql.catalog.lake=org.apache.iceberg.spark.SparkCatalog \
--conf spark.sql.catalog.lake.type=rest \
--conf spark.sql.catalog.lake.uri=https://{irc_host}/iceberg/ \
--conf spark.sql.catalog.lake.warehouse={catalog} \
--conf spark.sql.catalog.lake.token={token} \
--conf spark.sql.catalog.lake.header.X-Iceberg-Access-Delegation=vended-credentials
3. Grant access. Reading a table takes USE_CATALOG, USE_SCHEMA, and SELECT_TABLE; writing
takes MODIFY_TABLE in place of SELECT_TABLE. See Roles.
Deployment
The IRC runs inside the Gravitino server as an auxiliary service, on its own port, 9001 by default,
under the path /iceberg/. It is enabled when the server is deployed, and the UI shows its address
alongside the other client endpoints, or shows it as not enabled.
The service reads the metalake named by gravitino.iceberg-rest.gravitino-metalake and serves that
metalake's Iceberg catalogs. One deployment serves one metalake through the IRC.
Selecting a Catalog
The Iceberg REST specification carries the catalog in the request path as a prefix, as in
/iceberg/v1/{catalog}/namespaces. Clients do not build that path themselves: a client names the
catalog as its warehouse, calls /iceberg/v1/config with it, and receives the prefix to use for
every later request.
A client that names no warehouse gets the catalog set in gravitino.iceberg-rest.default-catalog-name.
Naming the warehouse explicitly is the safer habit.
Authentication and Access Control
Every request needs a credential. A request with none returns 401, and one with an expired token
returns 419, a status some Iceberg clients do not recognize as an expiry. Bearer tokens from the
identity provider and local accounts with Basic credentials are both accepted.
Authorization is the same as for the Gravitino API. The IRC passes the caller, the object, and the
operation to Gravitino, and the request proceeds only if the caller holds the privileges for it. A
denied request returns 403.
| Operation | Privileges |
|---|---|
| List namespaces | USE_CATALOG |
| List tables in a namespace | USE_CATALOG and USE_SCHEMA |
| Load a table | USE_CATALOG, USE_SCHEMA, and SELECT_TABLE or MODIFY_TABLE |
| Create a table | USE_CATALOG, USE_SCHEMA, and CREATE_TABLE |
| Commit to a table | USE_CATALOG, USE_SCHEMA, and MODIFY_TABLE |
Owning an object satisfies these checks as described in Roles.
Token Refresh
Some engines try to exchange or refresh a token in ways an identity provider does not support, which surfaces as authentication failures partway through a long job. Turning off token exchange avoids it:
# Spark
spark.sql.catalog.{catalog}.token-exchange-enabled=false
# Trino
iceberg.rest-catalog.oauth2.token-exchange-enabled=false
Credential Vending
An engine that sends X-Iceberg-Access-Delegation: vended-credentials receives short-lived storage
credentials with each table it loads, so it never holds long-lived storage keys of its own.
On S3 with the s3-token provider, the credential is scoped by a session policy to the table's own
location: it can read and write that table and nothing else in the bucket, even when the role behind
it can reach the whole account. The session is named gravitino_{principal}, so storage access logs
attribute each read and write to the Gravitino user who made it. A caller with MODIFY_TABLE
receives a credential that can write; a caller with only SELECT_TABLE receives one that can only
read.
See Credential Vending for providers and their properties.
Capabilities and Limits
The IRC supports namespace, table, and view operations, hierarchical namespaces, server-side scan
planning, and optimistic concurrency on commits, so a commit made against a stale table state is
rejected with 409 rather than overwriting a concurrent change.
| Not Supported | Effect |
|---|---|
| Multi-table transactions | Each commit applies to one table |
| View registration | Views are created through the IRC; existing views cannot be registered |
Views need the JDBC catalog backend with its schema at version V1, which is the default.
Configuration
| Configuration Item | Description | Default Value |
|---|---|---|
gravitino.iceberg-rest.gravitino-metalake | The metalake whose Iceberg catalogs the IRC serves | (none) |
gravitino.iceberg-rest.default-catalog-name | The catalog served to a client that names no warehouse | (none) |
gravitino.iceberg-rest.catalog-cache-eviction-interval-ms | How long a catalog's configuration is cached before being reloaded | 3600000 |
gravitino.iceberg-rest.table-metadata-cache-capacity | Tables whose metadata is cached | 1000 |
gravitino.iceberg-rest.table-metadata-cache-expire-minutes | How long cached table metadata is kept | 60 |
gravitino.iceberg-rest.scan-plan-cache-capacity | Scan plans cached for repeated identical queries | 200 |
gravitino.iceberg-rest.scan-plan-cache-expire-minutes | How long a cached scan plan is kept | 60 |
gravitino.iceberg-rest.jdbc-schema-version | Schema version of the JDBC backend; V1 enables views | V1 |
A change to a catalog's properties in Gravitino reaches the IRC when its cached configuration is next reloaded, up to the eviction interval later. Cached scan plans are keyed by snapshot, so a plan never outlives the table state it was computed for.
Health
The IRC answers its own health checks, which need no credential.
| Endpoint | Returns |
|---|---|
GET /iceberg/health/live | 200 whenever the service can answer |
GET /iceberg/health/ready | 200 once its catalogs are initialized, 503 before that |
GET /iceberg/health | 200 when both pass, 503 otherwise |
See Health and Readiness for the main server's checks.
Related
- Iceberg Catalog, for creating the catalogs the IRC serves
- Iceberg Federation, for serving a remote Iceberg REST catalog through Gravitino
- Credential Vending, for storage credentials
- Client Access, for the endpoints and connectors together