Skip to main content

Hive metastore as a Paimon catalog

Paimon tables registered in a Hive metastore are served by a Paimon catalog whose backend is that metastore. Connect the metastore as a Paimon catalog, and Gravitino lists, creates, alters and governs its Paimon tables. Nothing is moved or rewritten.

The same metastore can stay connected as a Hive catalog for its Hive tables and views, which hides the Paimon tables:

Catalog over the metastoreShows
Hive catalogHive tables and views; Paimon tables hidden
Paimon catalogPaimon tables, and also Hive tables Paimon can read directly, such as Parquet

Paimon lists Hive tables it can read as its own format tables. Govern Hive tables through the Hive catalog, so that grants and tags on them sit in one place.

Quick start​

The example connects a metastore that holds Paimon tables as the catalog hms_paimon and lists the tables in its database sales.

  1. Create the catalog. Set catalog-backend to hive and uri to the metastore's Thrift address. warehouse is where new tables are created.

    Connect the metastore as a Paimon catalog
    curl -sS -X POST "https://{gravitino_host}/api/metalakes/{metalake}/catalogs" \
      -H "Accept: application/vnd.gravitino.v1+json" \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $TOKEN" \
      -d '{
        "name": "hms_paimon",
        "type": "RELATIONAL",
        "provider": "lakehouse-paimon",
        "properties": {
          "catalog-backend": "hive",
          "uri": "thrift://{metastore_host}:9083",
          "warehouse": "s3://{bucket}/paimon"
        }
      }'

  2. List the tables.

    List the tables in a database
    curl -sS "https://{gravitino_host}/api/metalakes/{metalake}/catalogs/hms_paimon/schemas/sales/tables" \
      -H "Accept: application/vnd.gravitino.v1+json" \
      -H "Authorization: Bearer $TOKEN"

Properties​

The catalog takes the properties of any Paimon catalog, with catalog-backend set to hive, including the storage properties for the warehouse.

Access control​

Grants are made on each catalog's own objects. A grant on a schema in the Hive catalog does not cover the Paimon tables in the same metastore database; grant on the Paimon catalog's schema or tables for those. See Privileges.

Last updated on