StarRocks catalog

The StarRocks catalog connects Gravitino to a StarRocks cluster over the MySQL protocol, exposing StarRocks databases as schemas and StarRocks tables as tables. Schemas and tables are both read-write. One catalog reaches every database on the frontend named in jdbc-url.

Gravitino records its own identifier in the comment of each table it creates, as text of the form (From Gravitino, DO NOT EDIT: gravitino.v1.uid{id}), and uses it to recognize the table on later loads. Leave that text in place when you edit a table comment in StarRocks directly.

Quick start

The example connects a StarRocks frontend as the catalog starrocks_sales and lists its databases as schemas. The catalog uses MySQL Connector/J, which does not ship in the image. Enable it with gravitino.jdbcDrivers.mysql before you start, as described in JDBC driver.

my-values.yaml
gravitino:
  jdbcDrivers:
    enabled: true
    mysql:
      enabled: true

UI
REST

Start the catalog. Open Connect → Catalogs and click + Catalog. Enter starrocks_sales as the Catalog name, keep Relational as the Catalog type, and choose StarRocks under Databases and warehouses in Select provider.
Enter the connection. Under Connection, enter jdbc:mysql://{starrocks_fe_host}:9030 as the URL, and the User and Password Gravitino connects as. Advanced holds Cloud / region and a Properties list for any other catalog property. The UI fills in jdbc-driver itself.
Test and create it. Click Test connection, then Create catalog.
List the schemas. In Explore, open starrocks_sales. The schemas listed are the StarRocks databases, read from the frontend.

Create the catalog. The public REST endpoint does not fill in jdbc-driver, so the request names it.

Create a StarRocks catalog
curl -sS -X POST "https://{gravitino_host}/api/metalakes/{metalake}/catalogs" \
  -H "Accept: application/vnd.gravitino.v1+json" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "name": "starrocks_sales",
    "type": "RELATIONAL",
    "provider": "jdbc-starrocks",
    "comment": "StarRocks sales cluster",
    "properties": {
      "jdbc-url": "jdbc:mysql://{starrocks_fe_host}:9030",
      "jdbc-driver": "com.mysql.cj.jdbc.Driver",
      "jdbc-user": "{user}",
      "jdbc-password": "{password}"
    }
  }'

List the schemas. They are the StarRocks databases, read from the frontend.

List the schemas in the catalog
curl -sS "https://{gravitino_host}/api/metalakes/{metalake}/catalogs/starrocks_sales/schemas" \
  -H "Accept: application/vnd.gravitino.v1+json" \
  -H "Authorization: Bearer $TOKEN"

Catalog properties

The StarRocks catalog defines no catalog properties of its own; it takes the connection, pool, and driver properties every JDBC catalog shares, listed in Shared properties.

jdbc-url points at a frontend's MySQL protocol port, and jdbc-driver is com.mysql.cj.jdbc.Driver.

Driver version compatibility

The catalog requires Connector/J 8.0 or later, and catalog creation fails with an older driver, naming the version found. The chart's default Connector/J, 8.0.33, meets the requirement.

Schemas

A schema in this catalog is a StarRocks database. Create, list, load, and drop are supported; alter is not. Listing filters out information_schema.

Schema comments are rejected. Schema properties are written to the PROPERTIES clause of CREATE DATABASE, but a load returns no properties.

Dropping a schema that still holds tables fails unless the drop cascades.

warning

A cascading drop issues DROP DATABASE ... FORCE. StarRocks does not keep a database dropped with FORCE in its recycle bin, so the database and every table in it cannot be recovered with RECOVER.

Tables

Table create, load, list, rename, alter, and drop are supported. A drop issues DROP TABLE.

warning

A purge request does not drop the table. It issues TRUNCATE TABLE, which deletes every row and leaves the empty table in place. Use a drop to remove a table.

Properties

The catalog declares no table properties of its own. Every property in a create request is passed to the PROPERTIES clause of CREATE TABLE, and properties are read back from SHOW CREATE TABLE. StarRocks rejects a property it does not recognize, and Gravitino reports that as an illegal property.

An alter can set one property per request, issued as ALTER TABLE ... SET (...). Only these properties can be set:

Property	Notes
`replication_num`
`default.replication_num`
`default.storage_medium`
`enable_persistent_index`
`bloom_filter_columns`
`colocate_with`
`bucket_size`
`base_compaction_forbidden_time_ranges`
`dynamic_partition.{name}`	Any property with this prefix
`binlog.{name}`	Any property with this prefix

Removing a property is rejected.

Indexes

Indexes are not supported. Indexes in a create request are not written to StarRocks, and adding or deleting an index through an alter is rejected. A load reports each key StarRocks lists in SHOW INDEX as a primary key.

Partitioning and distribution

A table takes at most one partitioning transform, either range on one column or list on one or more columns. Composite partitioning and nested fields are rejected, and every partition column must be a column of the table. Partitions can be pre-assigned in the create request, and after creation they are listed, added, and dropped through the partitions API.

The distribution is optional. With none, Gravitino writes no DISTRIBUTED BY clause and StarRocks applies its default.

Gravitino distribution	StarRocks clause
`hash`	`DISTRIBUTED BY HASH({columns})`
`even`	`DISTRIBUTED BY RANDOM`

A bucket number of -1 omits the BUCKETS clause so StarRocks chooses the count. Any other number is written as BUCKETS {number}.

Sort orders are not supported.

Auto increment

An auto-increment column is written as AUTO_INCREMENT, and StarRocks applies its own rules to it. Changing auto increment through an alter is rejected.

Supported changes

All changes in one request other than a rename are issued as a single ALTER TABLE statement.

Change	Supported
Rename table	Yes, alone in its request
Update table comment	Yes
Add column	Yes, at any position; nullability and a default value are not applied
Delete column	Yes
Rename column	Yes, unless the new name is already a column
Update column type	Yes; the column's default is removed
Update column position	Yes
Update column comment	No
Update column nullable	No
Update column default	No
Update column auto increment	No
Add or delete index	No
Set property	Yes, one per request, from the list above
Remove property	No

Nested column names are rejected by every column change.

Identifier case and quoting

Names are case-sensitive in Gravitino and are passed to StarRocks exactly as given, never folded. Schema, table, and column names must match ^\w[\w/=-]{0,63}$: 1 to 64 characters, starting with an ASCII letter, digit, or underscore, and continuing with those or /, =, and -. StarRocks applies its own naming rules on top.

Gravitino quotes identifiers with backticks in generated SQL. Names returned by a list operation come back exactly as StarRocks stores them.

Type mapping

Gravitino reads each StarRocks type as the Gravitino type in its row, and creates each Gravitino type as the StarRocks type in its row, except where the list under the table says otherwise.

StarRocks type	Gravitino type
`BOOLEAN`, or `BIT` of size 1	Boolean
`TINYINT`	Byte
`SMALLINT`	Short
`INT`	Integer
`BIGINT`	Long
`FLOAT`	Float
`DOUBLE`	Double
`DECIMAL(p,s)`	Decimal(p, s)
`DATE`	Date
`DATETIME`	Timestamp without time zone
`CHAR(n)`	Char(n)
`VARCHAR(n)`	Varchar(n)
`VARCHAR(65533)`, `STRING`	String
`BINARY`, `VARBINARY`	Binary
Any other type	External type carrying the name the driver reports

StarRocks stores STRING as VARCHAR(65533), so a VARCHAR of exactly that length reads back as String. Types such as LARGEINT, JSON, ARRAY, MAP, STRUCT, BITMAP, and HLL come back as external types.

When Gravitino creates a column:

Boolean becomes boolean, String becomes string, and Binary becomes binary.
Char(n) takes n from 1 to 255, and Varchar(n) takes n from 1 to 1048576.
A timestamp's precision is not carried to StarRocks.
Any other Gravitino type is rejected, including external types, so a column cannot be created with a StarRocks type outside the table.

Column default values

CURRENT_TIMESTAMP is recognized as the current timestamp default in both directions. Literal defaults on numeric, decimal, date, datetime, char, and varchar columns are parsed into typed literals. Any other default, and any expression, is preserved as an unparsed expression. A function default other than CURRENT_TIMESTAMP is written inside parentheses.

Error mapping

StarRocks error codes are translated into Gravitino exceptions so a caller sees the same failure types it would from any other catalog.

StarRocks error	Gravitino exception
1007 database exists	Schema already exists
1050 table already exists	Table already exists
1008, 1049, 5501 database does not exist	No such schema
1051, 5502 unknown table	No such table
1045 access denied for user	Unauthorized
1054 unknown column	No such column
1507 partition does not exist	No such partition
1517 duplicate partition name	Partition already exists
5064 with "Unknown properties" in the message	Illegal property
Any other code with "Access denied" in the message	Connection failed

Any other error produces a generic runtime exception carrying the StarRocks message.

Quick start​

Catalog properties​

Driver version compatibility​

Schemas​

Tables​

Properties​

Indexes​

Partitioning and distribution​

Auto increment​

Supported changes​

Identifier case and quoting​

Type mapping​

Column default values​

Error mapping​

Related​