JDBC catalogs
Gravitino connects relational databases and warehouses through ten JDBC connectors: MySQL, PostgreSQL, Oracle, SQL Server, BigQuery, ClickHouse, Doris, StarRocks, OceanBase, and Hologres.
They share one property set for the connection, the connection pool, and create-time verification,
and each catalog loads a JDBC driver class named by jdbc-driver. Each connector page covers only
what it adds or constrains on top of the shared set.
Quick start
The example connects a MySQL server as the catalog mysql_sales and lists its databases as schemas.
The MySQL driver does not ship in the image, so supply it first, as described in
JDBC driver.
- UI
- REST
-
Start the catalog. Open Connect → Catalogs and click + Catalog. Enter
mysql_salesas the Catalog name, keep Relational as the Catalog type, and choose MySQL under Databases and warehouses in Select provider. -
Enter the connection. Under Connection, enter
jdbc:mysql://{mysql_host}:3306as the URL, and the User and Password Gravitino connects as. Advanced holds Cloud / region and a Properties list for any other property in Shared properties. The UI fills injdbc-driverfrom the provider. -
Test and create it. Click Test connection, then Create catalog. Gravitino tests the connection again before it saves the catalog.
-
List the schemas. In Explore, open
mysql_sales. The schemas listed are the MySQL databases, read from the server.
-
Create the catalog. The public REST endpoint does not fill in
jdbc-driver, so the request names it.Create a MySQL catalogcurl -sS -X POST "https://{gravitino_host}/api/metalakes/{metalake}/catalogs" \
-H "Accept: application/vnd.gravitino.v1+json" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{
"name": "mysql_sales",
"type": "RELATIONAL",
"provider": "jdbc-mysql",
"comment": "Sales database",
"properties": {
"jdbc-url": "jdbc:mysql://{mysql_host}:3306",
"jdbc-driver": "com.mysql.cj.jdbc.Driver",
"jdbc-user": "{user}",
"jdbc-password": "{password}"
}
}' -
List the schemas. They are the MySQL databases, read from the server.
List the schemas in the catalogcurl -sS "https://{gravitino_host}/api/metalakes/{metalake}/catalogs/mysql_sales/schemas" \
-H "Accept: application/vnd.gravitino.v1+json" \
-H "Authorization: Bearer $TOKEN"
Shared properties
| Property | Required | Default | Notes |
|---|---|---|---|
jdbc-url | Yes | The JDBC connection string | |
jdbc-driver | Yes | The driver class name. See JDBC driver for when it is filled in for you | |
jdbc-user | Yes | The user Gravitino connects as | |
jdbc-password | Yes | Hidden, not returned when the catalog is loaded. See the note below the table | |
jdbc-database | No | Required by some connectors; see below | |
jdbc.pool.min-size | No | 2 | Minimum idle connections in the pool |
jdbc.pool.max-size | No | 10 | Maximum connections in the pool |
jdbc.pool.test-on-borrow | No | true | Hidden. Validates a connection before use |
jdbc.pool.connect-timeout-ms | No | 30000 | Connect timeout in milliseconds, from 1 to 2147483647 |
verify-on-create | No | true | Hidden. Probes the source at create and fails the create if it cannot be reached |
A user holding USE_SECRETS and INCLUDE_CREDENTIAL_SECRETS on the catalog, or the metalake owner,
can retrieve jdbc-user and jdbc-password in plaintext, and the Spark and Flink connectors require
exactly that. Use the least-privileged account that covers what Gravitino must do on the source, and
read Catalog credentials before granting either
privilege.
jdbc-database is optional in the shared definition, but some connectors need a database. PostgreSQL
and SQL Server take it from jdbc-database or from jdbc-url (the path for PostgreSQL,
databaseName for SQL Server), and the create fails when neither names one or when the two differ.
Hologres requires jdbc-database itself. Read the connector page rather than assuming.
Pool sizing is per catalog, not per server. Ten JDBC catalogs at the default maximum can open a hundred connections against their sources, so check the total against the connection limits those sources enforce.
jdbc.pool.connect-timeout-ms is translated into each driver's own timeout parameter so an
unreachable source fails fast. Pooled connections get only the connect timeout, so long-running
operations are not cut off. The create-time probe also gets a read timeout. A timeout already set in
jdbc-url wins. The SQL Server connector does not translate it; set loginTimeout in jdbc-url
instead.
Set verify-on-create to false to register a catalog whose source is temporarily unavailable. It
skips only the connection probe. The driver and URL, boolean values and pool bounds are still checked
when the catalog is created or altered, without opening a connection, and an alter that fails those
checks leaves the stored configuration unchanged.
The probe holds only the catalog's own lock, so other catalogs in the metalake keep working while a slow source is probed.
A property with the gravitino.bypass. prefix is passed to the connection pool with the prefix
removed, so gravitino.bypass.maxWaitMillis sets the pool's maxWaitMillis.
JDBC driver
The UI fills in jdbc-driver from the provider when it is omitted, for both Test connection and
Create catalog. The public REST create endpoint, POST /api/metalakes/{metalake}/catalogs, and
the clients that call it do not, so REST and client requests must supply jdbc-driver. BigQuery is
the exception: its connector defaults jdbc-driver itself. A driver you supply is always kept.
If the driver class is not on the server classpath, the create fails and names the missing class.
| Connector | Provider | Driver class | In the image | Chart setting |
|---|---|---|---|---|
| MySQL | jdbc-mysql | com.mysql.cj.jdbc.Driver | No | gravitino.jdbcDrivers.mysql |
| PostgreSQL | jdbc-postgresql | org.postgresql.Driver | Yes | |
| Oracle | jdbc-oracle | oracle.jdbc.OracleDriver | Yes | |
| SQL Server | jdbc-sqlserver | com.microsoft.sqlserver.jdbc.SQLServerDriver | Yes | |
| BigQuery | jdbc-bigquery | com.simba.googlebigquery.jdbc42.Driver | No | gravitino.jdbcDrivers.bigquery |
| ClickHouse | jdbc-clickhouse | com.clickhouse.jdbc.ClickHouseDriver | Yes | |
| Doris | jdbc-doris | com.mysql.cj.jdbc.Driver | No | gravitino.jdbcDrivers.mysql |
| StarRocks | jdbc-starrocks | com.mysql.cj.jdbc.Driver | No | gravitino.jdbcDrivers.mysql |
| OceanBase | jdbc-oceanbase | com.mysql.cj.jdbc.Driver | No | gravitino.jdbcDrivers.mysql |
| Hologres | jdbc-hologres | org.postgresql.Driver | Yes |
The driver class shown is the one the UI fills in. OceanBase connects through MySQL Connector/J
with a jdbc:mysql:// URL; see OceanBase.
MySQL Connector/J is licensed under the GPL and the Simba BigQuery driver is not redistributed
either, so neither ships in the image. The chart supplies them at
start: it downloads each enabled driver from its default URL, verifies a pinned sha256, and links
it onto the classpath. Doris, StarRocks, and OceanBase use the same MySQL Connector/J as the MySQL
connector. Enable Connector/J only if you accept its license.
gravitino:
jdbcDrivers:
enabled: true
mode: download
mysql:
enabled: true
bigquery:
enabled: true
Enable only the drivers your catalogs use. A cluster without internet access sets
gravitino.jdbcDrivers.download.mavenMirror to an internal Maven mirror or
gravitino.jdbcDrivers.download.httpsProxy to a proxy, or sets gravitino.jdbcDrivers.mode to
image or volume to supply the jars itself. Downloaded jars are kept on a volume across restarts.