Skip to main content

Connect Snowflake to Iceberg REST

Snowflake reads Iceberg tables managed by Datastrato Enterprise through a Snowflake catalog integration pointed at the Iceberg REST service. Gravitino authorizes every request Snowflake makes and vends the storage credentials Snowflake uses to read the data files, so no Snowflake external volume is needed. Snowflake catalog integrations are read-only.

Prerequisites​

  • Iceberg REST service reachable from Snowflake over HTTPS. Snowflake connects from the public internet, so the service needs a public hostname and a certificate from a public authority. With the Helm chart, publish it through ingress.iceberg with TLS enabled.
  • Credential vending on the catalog. The Gravitino catalog Snowflake reads must vend credentials, for example credential-providers = s3-token with s3-role-arn set. See Credential vending.
  • An OAuth identity provider with an HTTPS token endpoint. Snowflake accepts only https:// token URIs. The Gravitino server must accept tokens from the same provider through the oauth authenticator.
  • A Gravitino user for the Snowflake identity. Gravitino maps the token to a user through gravitino.authenticator.oauth.principalFields. That user needs USE_CATALOG, USE_SCHEMA and SELECT_TABLE on what Snowflake reads.
  • A Snowflake role that can create integrations (ACCOUNTADMIN by default) and a running warehouse.

For best read performance, run the Snowflake account in the same cloud and region as the table data.

Create the catalog integration​

Create the catalog integration
CREATE OR REPLACE CATALOG INTEGRATION gravitino_iceberg
  CATALOG_SOURCE = ICEBERG_REST
  TABLE_FORMAT = ICEBERG
  CATALOG_NAMESPACE = 'sales'
  REST_CONFIG = (
    CATALOG_URI = 'https://iceberg.example.com/iceberg'
    CATALOG_NAME = 'lakehouse'
    ACCESS_DELEGATION_MODE = VENDED_CREDENTIALS
  )
  REST_AUTHENTICATION = (
    TYPE = OAUTH
    OAUTH_TOKEN_URI = 'https://idp.example.com/realms/datastrato/protocol/openid-connect/token'
    OAUTH_CLIENT_ID = 'snowflake'
    OAUTH_CLIENT_SECRET = '********'
    OAUTH_ALLOWED_SCOPES = ('profile')
  )
  ENABLED = TRUE;
SettingValue
CATALOG_URIThe Iceberg REST service base URL, ending in /iceberg. Snowflake appends /v1/config itself.
CATALOG_NAMEThe name of the Gravitino catalog. Gravitino returns it as the prefix for every later request.
CATALOG_NAMESPACEThe default schema for tables registered through this integration.
ACCESS_DELEGATION_MODEVENDED_CREDENTIALS, so Snowflake reads data with the scoped credentials Gravitino returns for each table.
OAUTH_ALLOWED_SCOPESThe scopes your identity provider requires for a client-credentials token.

To try the path before an HTTPS token endpoint is available, REST_AUTHENTICATION also accepts TYPE = BEARER with BEARER_TOKEN set to a token from your identity provider. The integration stops working when that token expires, so use it only for testing.

Check the connection
SELECT SYSTEM$VERIFY_CATALOG_INTEGRATION('gravitino_iceberg');

Register and query a table​

Register and query a table
CREATE DATABASE IF NOT EXISTS lakehouse_db;

CREATE OR REPLACE ICEBERG TABLE lakehouse_db.public.customers
  CATALOG = 'gravitino_iceberg'
  CATALOG_TABLE_NAME = 'customers';

SELECT * FROM lakehouse_db.public.customers;

Snowflake refreshes table metadata from Gravitino on the integration's REFRESH_INTERVAL_SECONDS, 30 seconds by default.

Access control​

Gravitino decides every request. When Gravitino denies the Snowflake identity access to a table, including a table Snowflake has already registered, the query fails with Gravitino's message, for example:

Error when Gravitino denies access
Resource on the REST endpoint of catalog integration GRAVITINO_ICEBERG is forbidden due to error:
Forbidden: User 'svc-snowflake' is not authorized to load table 'acme.lakehouse.sales.customers'.

Access returns as soon as the grant is restored. Manage the Snowflake identity's privileges in Gravitino under Authorize, the same as any other user.

Limitations​

  • Read-only: Snowflake cannot create or write tables through a catalog integration.
  • Iceberg catalogs only. Catalogs of other types are not served by the Iceberg REST service.
  • The OAuth token endpoint must be https://.
Last updated on