Connect Snowflake to Iceberg REST
Snowflake reads and writes Iceberg tables managed by Datastrato Enterprise through a Snowflake catalog integration pointed at the Iceberg REST service. Gravitino authorizes every request Snowflake makes and vends the storage credentials Snowflake uses to reach the data files, so no Snowflake external volume is needed unless the tables are shared onward to other engines. No Gravitino engine connector is involved; the engine connectors are only for the Gravitino REST interface.
A catalog integration exposes tables to Snowflake in one of two ways:
- A catalog-linked database (CLD) mirrors a whole Gravitino catalog. Snowflake discovers its schemas and tables, keeps them in sync, and can write to them.
- Individually registered tables expose one table at a time, read-only.
Quick start
The example signs in with OAuth client credentials, links the Gravitino catalog sales_lake as the
Snowflake database sales_lake_db, and reads its table q4.leads. The token endpoint must be https://.
CREATE OR REPLACE CATALOG INTEGRATION gravitino_iceberg
CATALOG_SOURCE = ICEBERG_REST
TABLE_FORMAT = ICEBERG
REST_CONFIG = (
CATALOG_URI = 'https://{iceberg_host}/iceberg'
CATALOG_NAME = 'sales_lake'
ACCESS_DELEGATION_MODE = VENDED_CREDENTIALS
)
REST_AUTHENTICATION = (
TYPE = OAUTH
OAUTH_TOKEN_URI = '{token_endpoint_uri}'
OAUTH_CLIENT_ID = '{client_id}'
OAUTH_CLIENT_SECRET = '{client_secret}'
OAUTH_ALLOWED_SCOPES = ('{scope}')
)
ENABLED = TRUE;
SELECT SYSTEM$VERIFY_CATALOG_INTEGRATION('gravitino_iceberg');
CREATE DATABASE sales_lake_db
LINKED_CATALOG = ( CATALOG = 'gravitino_iceberg' );
SELECT SYSTEM$CATALOG_LINK_STATUS('sales_lake_db');
SELECT * FROM sales_lake_db."q4"."leads" LIMIT 10;
The catalog must vend credentials for the read to reach storage; see Credential vending.
Prerequisites
- Iceberg REST service reachable from Snowflake over HTTPS. Snowflake connects from the public
internet, so the service needs a public hostname and a certificate from a public authority. With the
Helm chart, publish it through
ingress.icebergwith TLS enabled, as described in Client access. - Table data in Amazon S3, Google Cloud Storage or Azure Storage. Snowflake does not accept vended credentials for S3-compatible stores.
- Credential vending on the catalog. The Gravitino catalog Snowflake reads must vend credentials,
for example
credential-providers = s3-tokenwiths3-role-arnset. See Credential vending. - An OAuth identity provider with an HTTPS token endpoint.
Snowflake accepts only
https://token URIs. The Gravitino server must accept tokens from the same provider through theoauthauthenticator, so the token's audience must matchgravitino.authenticator.oauth.serviceAudience. - A Gravitino user for the Snowflake service account, with privileges. Gravitino maps the token to a
user through
gravitino.authenticator.oauth.principalFields. The user appears under Authorize after Snowflake's first request, which Gravitino refuses until a role grants access. Add the user to a role withUSE_CATALOG,USE_SCHEMAandSELECT_TABLEon what Snowflake reads, plusMODIFY_TABLEon what it writes. - A Snowflake role that can create integrations and databases (
ACCOUNTADMINby default) and a running warehouse.
For best read performance, run the Snowflake account in the same cloud and region as the table data.
Create the catalog integration
The CREATE CATALOG INTEGRATION statement in the Quick start is the complete form.
Its settings:
| Setting | Value |
|---|---|
CATALOG_URI | The Iceberg REST service base URL, ending in /iceberg. Snowflake appends /v1/config itself. |
CATALOG_NAME | The name of the Gravitino catalog. Gravitino returns it as the prefix for every later request. |
CATALOG_NAMESPACE | Optional. The default schema for tables registered individually through this integration. |
ACCESS_DELEGATION_MODE | VENDED_CREDENTIALS, so Snowflake reads data with the scoped credentials Gravitino returns for each table. |
OAUTH_ALLOWED_SCOPES | The scopes your identity provider requires for a client-credentials token. |
REFRESH_INTERVAL_SECONDS | Optional. How often Snowflake polls Gravitino for table changes, 30 seconds by default. |
REST_AUTHENTICATION also accepts TYPE = BEARER with BEARER_TOKEN set to a
token from your identity provider. Gravitino validates a
bearer token exactly as it validates one Snowflake obtains through OAuth, but the integration stops
working when the token expires. Replace it before then:
ALTER CATALOG INTEGRATION gravitino_iceberg SET REST_AUTHENTICATION = (
TYPE = BEARER
BEARER_TOKEN = '{new_token}'
);
SELECT SYSTEM$VERIFY_CATALOG_INTEGRATION('gravitino_iceberg');
Link a whole catalog
A catalog-linked database turns each namespace of the Gravitino catalog into a Snowflake schema and each Iceberg table into a Snowflake Iceberg table, and picks up tables created later without any further statements.
CREATE DATABASE {database}
LINKED_CATALOG = ( CATALOG = 'gravitino_iceberg' );
SELECT SYSTEM$CATALOG_LINK_STATUS('{database}');
SHOW SCHEMAS IN DATABASE {database};
SHOW ICEBERG TABLES IN DATABASE {database};
Snowflake folds unquoted identifiers to uppercase, so quote the lowercase schema, table and column names that come from Gravitino:
SELECT * FROM {database}."{schema}"."{table}" WHERE "{column}" = 4;
Snowflake discovers new schemas and tables on the database's SYNC_INTERVAL_SECONDS and picks up new
snapshots of tables it already knows on the integration's REFRESH_INTERVAL_SECONDS. Set either with
ALTER DATABASE ... UPDATE LINKED_CATALOG or ALTER CATALOG INTEGRATION.
Writes
Writes through a catalog-linked database commit to Gravitino, so other engines see them on their next read:
INSERT INTO {database}."{schema}"."{table}" VALUES (4, 'fresh');
The credentials Gravitino vends follow the Snowflake service account's privileges. Without MODIFY_TABLE on
the table, Gravitino vends read-only storage credentials and the write fails when Snowflake writes the
data file:
Failed to access remote file: access denied. Please check your credentials
A grant in Gravitino applies to the next statement Snowflake runs.
Register and query a single table
To expose one table without linking the catalog, register it as a Snowflake Iceberg table that names
the integration and the table's name in the Gravitino catalog. Set CATALOG_NAMESPACE on the
integration to the table's schema.
CREATE OR REPLACE ICEBERG TABLE {database}.public.{table}
CATALOG = 'gravitino_iceberg'
CATALOG_TABLE_NAME = '{table}';
SELECT * FROM {database}.public.{table};
Access control
Snowflake does not pass its users' identities to Gravitino. Every request from the integration reaches Gravitino as the one Snowflake service account, so access control has two layers:
| Layer | Grant to | Controls |
|---|---|---|
| Gravitino | The Snowflake service account | Everything Snowflake can reach in the catalog, for all of its users together |
| Snowflake | Individual Snowflake roles | Which Snowflake users see which schemas and tables of the linked database |
Grant the service account privileges in Gravitino under Authorize, covering everything any Snowflake user needs, and grant individual access with Snowflake grants on the linked database. Gravitino's privileges for individual people do not apply to queries that come through Snowflake.
When Gravitino denies the service account access to a table, including a table Snowflake has already registered, the query fails with Gravitino's message, for example:
Resource on the REST endpoint of catalog integration GRAVITINO_ICEBERG is forbidden due to error:
Forbidden: User 'svc-snowflake' is not authorized to load table 'acme.lakehouse.sales.customers'.
Access returns as soon as the grant is restored.
Share linked tables with other engines through Snowflake
Snowflake Horizon Catalog serves the tables of a catalog-linked database to outside engines, and
BigQuery reaches Gravitino tables that way. Horizon passes those engines storage credentials only from a
Snowflake external volume, never credentials Gravitino vended, so a catalog-linked database meant for
other engines uses an integration with ACCESS_DELEGATION_MODE = EXTERNAL_VOLUME_CREDENTIALS and a
database created with EXTERNAL_VOLUME. The full setup is in
Connect BigQuery through Snowflake.
Limitations
- Iceberg views are not synced. A catalog-linked database exposes the catalog's tables only, even for views that carry a Snowflake SQL representation. Define the views in Snowflake over the linked tables instead.
- Individually registered tables are read-only. Use a catalog-linked database to write.
- Iceberg catalogs only. Catalogs of other types are not served by the Iceberg REST service.
- The OAuth token endpoint must be
https://.