Google Cloud Storage

An Iceberg catalog on Google Cloud Storage keeps its tables under a gs:// warehouse. The server reaches the bucket as a Google service account, and the Iceberg REST service vends engines access tokens downscoped to each table.

Quick start

The example creates the catalog iceberg_gcs with its warehouse at gs://{bucket}/warehouse, a PostgreSQL backend, and tokens vended through gcs-token. The server uses a service account key file mounted into its pod.

Mount the key file. Store the service account's JSON key in a Secret and mount it into the server pod, as described in Google Cloud credentials. That puts the key at /etc/gcs/key.json. To use Workload Identity instead of a key file, see Without a key file.

UI
REST

Start the catalog. Open Connect → Catalogs and click Catalog. Enter iceberg_gcs as the Catalog name, choose Relational as the Catalog type, and Apache Iceberg as the provider.

Fill in the connection.

Field	Value
Catalog backend	`jdbc`
URI	`jdbc:postgresql://{postgres_host}:5432/{database}`
User, Password	The backend database's user and password
Storage	GCS
Warehouse	`gs://{bucket}/warehouse`
Service account file	`/etc/gcs/key.json`

Create it. Click Test connection, which checks the backend, then Create catalog. With a service account file set, the catalog vends tokens through gcs-token with nothing more to add.

Create the catalog.

Create an Iceberg catalog on GCS
curl -sS -X POST "https://{gravitino_host}/api/metalakes/{metalake}/catalogs" \
  -H "Accept: application/vnd.gravitino.v1+json" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "name": "iceberg_gcs",
    "type": "RELATIONAL",
    "provider": "lakehouse-iceberg",
    "properties": {
      "catalog-backend": "jdbc",
      "uri": "jdbc:postgresql://{postgres_host}:5432/{database}",
      "jdbc-driver": "org.postgresql.Driver",
      "jdbc-user": "{jdbc_user}",
      "jdbc-password": "{jdbc_password}",
      "warehouse": "gs://{bucket}/warehouse",
      "gcs-service-account-file": "/etc/gcs/key.json",
      "credential-providers": "gcs-token"
    }
  }'

Over this API, jdbc-driver has to be set; the UI fills it in from the URI.

Connect an engine. Point it at the Iceberg REST service with iceberg_gcs as the warehouse, and have it ask for vended credentials. See Spark or Trino.
Check it. Create a table, insert a row, and read it back. Test connection never touches the bucket.

Catalog properties

Property	Meaning	Default
`warehouse`	`gs://{bucket}/{path}`	(none)
`gcs-service-account-file`	Path, on the server, to the service account's JSON key. The server uses it for its own access and for `gcs-token`	Application default credentials

gcs-service-account-file is a path on the server's file system, not the key itself, so the file has to be mounted into the server pod. The property is not hidden, but the key it points to never leaves the server.

Credentials for engines

Provider	Engines receive
`gcs-token`	An OAuth access token, downscoped to the table
(none)	Nothing; engines bring their own credentials

There is no provider that vends a static key for GCS.

`gcs-token`

The server downscopes its own credentials with a credential access boundary and returns the token, with its expiry and an endpoint to refresh it from. The boundary allows reading and listing under the table's location, and writing for a caller who may modify the table.

Grant the service account Storage Object User (roles/storage.objectUser) on the bucket. A downscoped token can never exceed what the service account itself holds.

A catalog with gcs-service-account-file set and no credential-providers uses gcs-token automatically. Without the file, set credential-providers to gcs-token.

Without a key file

On GKE with Workload Identity, leave gcs-service-account-file unset. The server then uses its pod's Google identity, both for its own access and as the source of the tokens it vends. Bind the server's Kubernetes service account to a Google service account with gravitino.serviceAccount.annotations, and set credential-providers to gcs-token on the catalog.

Troubleshooting

HTTP 400 "No credential provider found for path": the engine asked for vended credentials and the catalog has no GCS provider. A catalog without a service account file needs credential-providers set explicitly.

The server cannot read the key file: the file is not mounted at the path in gcs-service-account-file, or the Secret's key name does not match the file name.

Tokens are vended but reads fail with 403: the service account lacks a storage role on the bucket.

Quick start​

Catalog properties​

Credentials for engines​

gcs-token​

Without a key file​

Troubleshooting​

Related​