Skip to main content

Statistics

Statistics describe the data in a table: how many files it has, how large they are, how many are delete files. Gravitino stores them on the table, or on each of its partitions, and returns them through the API.

In 1.3, statistics serve table maintenance. The built-in job builtin-iceberg-update-stats reads an Iceberg table's file metadata and writes the statistics below, and compaction policies read them to decide when a table needs compacting. Gravitino computes no statistics on its own, engines do not read them, and the UI does not show them. You can also record statistics of your own for your own tools, as in Record your own statistics.

Statistics are not metrics. Metrics measure the Gravitino server itself, such as requests, threads, and memory, and are scraped by Prometheus. Statistics describe the data in your tables and are read through the API.

Quick start​

  1. Compute the statistics. Run the builtin-iceberg-update-stats template on the table from Automate → Jobs → Run job; see Update statistics.

  2. Read them.

    List a table's statistics
    curl -sS "https://{gravitino_host}/api/metalakes/{metalake}/objects/table/{catalog}.{schema}.{table}/statistics" \
      -H "Accept: application/vnd.gravitino.v1+json" \
      -H "Authorization: Bearer $TOKEN"

    The response lists each statistic with its value and who set it when. For a partitioned table the job writes per partition, so list them with the partition statistics call instead.

Statistics the update job writes​

The job counts the files in each partition, or in the whole table when it is not partitioned.

StatisticTypeWhat it measures
custom-file-numberIntegerAll files
custom-data-file-numberIntegerData files
custom-delete-file-numberIntegerPosition and equality delete files together
custom-position-delete-file-numberIntegerPosition delete files
custom-equality-delete-file-numberIntegerEquality delete files
custom-small-file-numberIntegerFiles smaller than 32 MiB
custom-data-file-mseDecimalMean squared shortfall of data files below 128 MiB, in bytes²
custom-avg-file-sizeDecimalAverage file size, in bytes
custom-total-file-sizeIntegerTotal size of all files, in bytes

Compaction policies read custom-data-file-mse and custom-delete-file-number. The statistics are a snapshot from the job's last run: a compaction or a write does not refresh them, so run the job again before a policy evaluates the table.

Record your own statistics​

You can attach statistics of your own to any table, for tools that track something about it, such as a row count from your pipeline or the date a table was last validated. Gravitino stores each value and returns it with the table's other statistics; nothing in Gravitino acts on it.

  • Name. Start every name with custom-, and keep it to 128 characters. Names without the prefix are rejected.
  • Value. Send the value as JSON. A number, string, boolean, list, or object keeps its type, so 1000 is stored as a number and "1000" as a string.
  • Who can write. Writing a statistic takes the privilege to modify the table; reading takes the privilege to read it.

  1. Set one or more statistics. Writing a name that already exists replaces its value.

    Set table statistics
    curl -sS -X PUT "https://{gravitino_host}/api/metalakes/{metalake}/objects/table/{catalog}.{schema}.{table}/statistics" \
      -H "Accept: application/vnd.gravitino.v1+json" \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $TOKEN" \
      -d '{"updates": {"custom-row-count": 1000, "custom-validated-on": "2026-10-18"}}'

  2. Read them with the call in the Quick start. Your statistics appear beside any the update job wrote.

  3. Remove one by name.

    Drop a table statistic
    curl -sS -X POST "https://{gravitino_host}/api/metalakes/{metalake}/objects/table/{catalog}.{schema}.{table}/statistics" \
      -H "Accept: application/vnd.gravitino.v1+json" \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $TOKEN" \
      -d '{"names": ["custom-validated-on"]}'

To record figures per partition instead, use the partition statistics calls, which take the partition names and write several partitions in one request; see Partition statistics.

The statistics model​

Reserved and custom​

A reserved statistic is one Gravitino defines, such as row_count. The set can grow in future releases, so reserved names are not available for your own use.

A custom statistic is yours, and its name must begin with custom- to keep it clear of anything Gravitino may reserve later. Names are limited to 128 characters. Every statistic written through the API, for a table or a partition, must be a custom one.

Modifiable and not​

A statistic is marked modifiable or not. Gravitino rejects a write to a statistic that is not modifiable rather than accept a value it would later replace. Custom statistics are modifiable, and in 1.3 every statistic a table carries is a custom one, written through the API or by a job.

Values​

A value is typed rather than free text: a number, a string, a boolean, a list, or an object. A row count sent as 1000 is stored as a number, while "1000" is stored as a string. A statistic cannot be written without a value.

Partition statistics​

Statistics attach to partitions as well as to whole tables. The update job writes per partition on a partitioned table, so each partition carries its own figures.

Partition statistics are held by Gravitino rather than by the catalog, so they work on any table, including tables in catalogs that expose no partition objects. Partition names are supplied by the caller, several partitions are written in one request, and statistics are listed over a range of partition names rather than one at a time. Where they are stored is a server setting; see Partition statistics in the server configuration.

Permissions​

Statistics follow the table they belong to. Reading them requires the privilege to read the table, and writing them requires the privilege to modify it. See Tables and views.

Use the API​

Statistics can be listed, set, and dropped over REST and through the Java client, for tables and for partitions. The Python client covers table statistics but not partition statistics. The REST reference documents each call for table statistics and partition statistics, with the client code under the Java and Python tabs.

Last updated on