Skip to main content

Connecting a Catalog

A catalog connects Gravitino to a system that already holds metadata, such as a lakehouse, a metastore, or a relational database. Gravitino does not copy that metadata. It reads and writes through to the source, so the source stays authoritative and a change made outside Gravitino is visible the next time the catalog is read.

The pages in this section cover the properties each specific system needs. This page covers what they have in common: how a connection is validated, how properties behave, and what happens when a catalog cannot be reached.

Quick Start​

1. Create a catalog. Every catalog declares a name, a type, a provider, and a property map. The provider selects the connector, and the properties configure it.

GRAVITINO_URL=http://localhost:8090
METALAKE=metalake_demo

curl -X POST -H "Content-Type: application/json" \
  -d '{
    "name": "{catalog_name}",
    "type": "RELATIONAL",
    "provider": "{provider}",
    "comment": "",
    "properties": {}
  }' \
  "${GRAVITINO_URL}/api/metalakes/${METALAKE}/catalogs"

2. Confirm it connected. Creation fails outright if the connection cannot be established, so a successful response means the source answered.

3. List the schemas. What comes back is read from the source rather than from Gravitino's own store.

Connection Testing at Creation​

Gravitino tests the connection before it commits the catalog. It builds a throwaway catalog instance from the properties you supplied, asks the connector to test the connection, and closes it again. Only then is the catalog entity written.

A wrong host, a bad credential, or a missing driver therefore surfaces immediately as a failed create rather than as a working catalog that fails on first use. The error comes from the connector and names the underlying cause.

Property Behavior​

Properties fall into four categories, and a connector's page states which of its own properties fall where.

Required properties must be supplied at creation. Optional properties have defaults that apply when omitted. Immutable properties are fixed at creation and cannot be changed by an alter. Reserved properties are set by Gravitino rather than by you, and are rejected if you try to supply them.

A property may also be hidden, which affects reading rather than writing. A hidden property is accepted at creation and used by the connector, but is not returned when the catalog is loaded. Credentials are hidden for this reason, so a password you set is never echoed back.

Beyond a connector's own properties, every catalog accepts a small set of common ones covering the classpath package to load the connector from, an override for the catalog operations implementation, an authorization provider, and the cloud and region the catalog runs in. Gravitino also maintains reserved properties recording whether the catalog and its metalake are in use.

JDBC Properties​

The relational connectors under Connect share one property set, so the table below applies to MySQL, PostgreSQL, Oracle, SQL Server, ClickHouse, Doris, StarRocks, OceanBase, and Hologres alike. A connector's own page covers only what it adds or constrains on top of this.

PropertyRequiredDefaultNotes
jdbc-urlYesThe JDBC connection string
jdbc-driverYesThe driver class name
jdbc-userYesHidden, not returned when the catalog is loaded
jdbc-passwordYesHidden, not returned when the catalog is loaded
jdbc-databaseNoRequired by some connectors even though the shared definition treats it as optional
jdbc.pool.min-sizeNo2Minimum pooled connections
jdbc.pool.max-sizeNo10Maximum pooled connections
jdbc.pool.test-on-borrowNotrueHidden, validates a connection before use

jdbc-database is where the shared definition and the individual connectors disagree, so read the connector page rather than assuming. SQL Server requires it and fails at initialization without it. Connectors that address a schema directly do not use it at all.

Pool sizing is per catalog, not per server. A metalake holding ten JDBC catalogs at the default maximum can open a hundred connections against its sources, which is worth checking against the connection limits those sources enforce.

Catalog Lifetime and Caching​

A connected catalog is cached in memory along with its connection pool and its isolated classloader. The cache evicts on an access interval rather than on a fixed lifetime, so a catalog nobody has touched is closed and its connections released, and the next request reopens it.

The practical consequence is that the first request to an idle catalog pays reconnection cost. A source that is slow to accept connections shows this as intermittent latency on otherwise identical requests.

Each catalog loads its connector in an isolated classloader, which is why two catalogs can use different, incompatible driver versions of the same database in one server.

When a Catalog Cannot Be Reached​

A catalog whose source has become unreachable stays defined. Gravitino holds the catalog entity in its own store, so the catalog continues to appear in listings and its properties remain readable while operations against it fail.

The failure surfaces per operation and carries the connector's own error. Listing schemas on an unreachable catalog fails; listing catalogs in the metalake does not.

A catalog can also be disabled deliberately, which is distinct from being unreachable. A disabled catalog rejects operations because Gravitino refuses them, not because the source is down, and disabling a metalake disables all its catalogs.

Last updated on