Connect StarRocks to Iceberg REST

Apache Gravitino exposes an Iceberg REST catalog (IRC) endpoint that any Iceberg-compatible engine can connect to directly. StarRocks connects to it with an external Iceberg catalog of type rest. No Gravitino engine connector is needed; the engine connectors are only for the Gravitino REST interface.

Quick start

The example signs in with OAuth client credentials, connects to the Gravitino catalog sales_lake as the StarRocks catalog sales, and reads storage through vended credentials, which needs StarRocks 4.0 or later.

Create the catalog and query a table
CREATE EXTERNAL CATALOG sales
PROPERTIES
(
  "type"                                       = "iceberg",
  "iceberg.catalog.type"                       = "rest",
  "iceberg.catalog.uri"                        = "https://{iceberg_host}/iceberg",
  "iceberg.catalog.warehouse"                  = "sales_lake",
  "iceberg.catalog.security"                   = "OAUTH2",
  "iceberg.catalog.oauth2.credential"          = "{client_id}:{client_secret}",
  "iceberg.catalog.oauth2.server-uri"          = "{token_endpoint_uri}",
  "iceberg.catalog.oauth2.scope"               = "{scope}",
  "iceberg.catalog.vended-credentials-enabled" = "true"
);

SHOW DATABASES FROM sales;
SELECT * FROM sales.q4.leads LIMIT 10;

The catalog must vend credentials for the read to reach storage; see Credential vending.

Prerequisites

Apache Gravitino running with the Iceberg REST service enabled. See Iceberg REST catalog service for setup instructions.
The Iceberg REST service published and reachable from your StarRocks environment. See Client access.

Configuration

The catalog in the Quick start connects with OAuth client credentials and reads storage through vended credentials, which StarRocks supports from 4.0. Every request to the IRC needs an identity; a request with none is rejected with 401.

iceberg.catalog.warehouse is the name of the Gravitino catalog to connect to, not a storage path. Without it, the service serves its default catalog, if one is set; see Select a catalog. To use a token you already hold, set iceberg.catalog.oauth2.token in place of the credential, server URI and scope. Gravitino catalogs with nested namespaces also need iceberg.catalog.rest.nested-namespace-enabled set to true.

On StarRocks earlier than 4.0, which cannot use vended credentials, give StarRocks its own storage keys instead:

Static storage keys
  "aws.s3.access_key"               = "{access_key}",
  "aws.s3.secret_key"               = "{secret_key}",
  "aws.s3.endpoint"                 = "https://{s3_host}:9000",
  "aws.s3.enable_path_style_access" = "true",
  "client.factory"                  = "com.starrocks.connector.iceberg.IcebergAwsClientFactory"

client.factory must be set explicitly with static keys for StarRocks to initialize the Iceberg AWS client correctly.

Examples

Create, write and query a table
SET CATALOG sales;
CREATE DATABASE db;
USE db;
CREATE TABLE t(a int);
INSERT INTO t VALUES (1);
SELECT * FROM t;

Gravitino connector vs. Iceberg REST

Feature	Gravitino engine connector	Iceberg REST
Engine plugin required	Yes	No
Gravitino access control	Yes	Yes
Supported engines	Trino, Spark, Flink, Daft	Any Iceberg-compatible engine
Credential vending	Varies	Yes (S3, GCS, OSS, ADLS)

Quick start​

Prerequisites​

Configuration​

Examples​

Gravitino connector vs. Iceberg REST​

Related​

Quick start

Prerequisites

Configuration

Examples

Gravitino connector vs. Iceberg REST

Related