Skip to main content

Partitions

A partition is a named slice of a table's data, and in Gravitino it is a metadata object you can list, inspect, add, and remove.

Most of the time partitions look after themselves. A catalog creates them as data arrives and an engine prunes them at query time without anyone asking. Managing them by hand matters in a few specific cases: expiring data by partition rather than by row, recording statistics per partition so a planner can prune on them, and adding a partition ahead of the data that will fill it.

Partitions belong to partitioned tables. A table becomes partitioned when it is created with a partitioning strategy, which is a separate thing from the partitions themselves. See Table partitioning, distribution, sort order, and indexes.

Quick start​

The example manages the partitions of the table orders in sales.public, partitioned by its dt column. Partitions are managed through the API or an engine; the UI does not show them. Which path applies depends on the catalog.

On these catalogs, partitions are objects the Gravitino API lists, adds, and drops.

  1. List the partitions. The response gives each partition's name; ?details=true returns the field names, values, and properties as well.

    List a table's partitions
    curl -sS "https://{gravitino_host}/api/metalakes/example/catalogs/sales/schemas/public/tables/orders/partitions" \
      -H "Accept: application/vnd.gravitino.v1+json" \
      -H "Authorization: Bearer $TOKEN"

  2. Add a partition. An identity partition gives its field names and values in the same order. Hive derives the partition's name, here dt=2026-10-01, from them.

    Add a partition
    curl -sS -X POST "https://{gravitino_host}/api/metalakes/example/catalogs/sales/schemas/public/tables/orders/partitions" \
      -H "Accept: application/vnd.gravitino.v1+json" \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $TOKEN" \
      -d '{
      "partitions": [{
        "type": "identity",
        "fieldNames": [["dt"]],
        "values": [{"type": "literal", "dataType": "string", "value": "2026-10-01"}]
      }]
    }'

  3. Look at it.

    Get a partition
    curl -sS "https://{gravitino_host}/api/metalakes/example/catalogs/sales/schemas/public/tables/orders/partitions/dt=2026-10-01" \
      -H "Accept: application/vnd.gravitino.v1+json" \
      -H "Authorization: Bearer $TOKEN"

  4. Drop it. Dropping removes the partition from the catalog.

    Drop a partition
    curl -sS -X DELETE "https://{gravitino_host}/api/metalakes/example/catalogs/sales/schemas/public/tables/orders/partitions/dt=2026-10-01" \
      -H "Accept: application/vnd.gravitino.v1+json" \
      -H "Authorization: Bearer $TOKEN"

The partition model​

Partition types​

TypeDescribes
IDENTITYOne value per partitioning column, such as dt=2026-08-02
RANGEAn upper and lower bound, such as a date range
LISTAn explicit set of value combinations

The type follows from how the table was partitioned rather than being chosen per partition. Hive and Glue accept identity partitions only.

An identity partition lists its fieldNames and its values, and the values must be in the same order as the field names. On Hive, the partition name in the request is ignored, and Hive derives the name from the field names and values.

Dropping a partition removes it from the catalog. The purge option on a drop is not supported by any catalog today and is rejected.

Where partition management works​

Partition management through the Gravitino API works on Hive, Glue, Doris, StarRocks, and MaxCompute catalogs, and on no others.

Iceberg, Paimon, and Hudi manage partitions themselves, and the other JDBC catalogs have no partition operations, so a call against one of them fails rather than returning an empty list. Nothing is missing from Gravitino for those catalogs, and the sections below say what to do instead.

Iceberg tables​

Iceberg partitions differently, and the difference is deliberate rather than a gap.

An Iceberg table declares a partition spec, a set of transforms over its columns such as day(event_time) or bucket(16, customer_id). Partitions are then derived from the data as it arrives, which Iceberg calls hidden partitioning. There is no partition object to add, get, or drop, which is why Iceberg is not in the list above.

Gravitino sets the spec when it creates the table. Identity, bucket, truncate, year, month, day, and hour transforms all convert to an Iceberg spec, with bucket limited to one field. List and range partitioning have no Iceberg equivalent and are rejected.

After creation, partitioning is changed through the engine rather than through the Gravitino API, using the engine's own syntax, as in the Iceberg tab of the Quick start.

The Iceberg REST catalog service handles the resulting AddPartitionSpec and SetDefaultPartitionSpec updates, so partition evolution behaves exactly as it does against any Iceberg catalog. Existing data is not rewritten, and the new spec applies to what is written afterward.

See Iceberg REST catalog service.

Partitions and statistics​

Statistics attach to partitions as well as to whole tables, which is what makes per-partition measurements available to a planner. See Statistics.

Partitions in the UI​

The UI does not show a table's partitions or its partitioning. Use the API, or for Iceberg an engine, as in the Quick start.

Permissions​

Partitions follow the table they belong to. Listing them requires the privilege to read the table, and adding or dropping requires the privilege to modify it. See Tables and views.

Use the API​

On Hive, Glue, Doris, StarRocks, and MaxCompute, partitions are listed, added, inspected, and dropped over REST and through the Java and Python clients. The REST reference documents each call, with the client code under the Java and Python tabs.

On Iceberg, partitioning is changed through an Iceberg-compatible engine pointed at the Iceberg REST catalog service rather than through these endpoints.

Partition statistics are a separate path and work on any table, including Iceberg. They are held by Gravitino rather than by the catalog, so a partition can carry statistics even where the catalog exposes no partition objects to list. Partition names are supplied by the caller rather than discovered, and statistics are read and written over a range rather than one at a time. See Statistics.

Last updated on