CLI Reference

Overview

Use --help to list all commands, or --help --type <command> for command-specific help.

By default, the CLI loads conf/gravitino-optimizer.conf from the current working directory. Use --conf-path only when you need a custom config file.

Command Quick Reference

Command (`--type`)	Required options	Optional options	Purpose
`submit-strategy-jobs`	`--identifiers`, `--strategy-name`	`--dry-run`, `--limit`	Recommend and optionally submit jobs
`update-statistics`	`--calculator-name`	`--identifiers`, `--statistics-payload`, `--file-path`	Calculate and persist statistics
`append-metrics`	`--calculator-name`	`--identifiers`, `--statistics-payload`, `--file-path`	Calculate and append metrics
`monitor-metrics`	`--identifiers`, `--action-time`	`--range-seconds`, `--partition-path`	Evaluate rules with before/after metrics
`list-table-metrics`	`--identifiers`	`--partition-path`	Query stored table or partition metrics
`list-job-metrics`	`--identifiers`	None	Query stored job metrics
`submit-update-stats-job`	`--identifiers`	`--dry-run`, `--update-mode`, `--updater-options`, `--spark-conf`	Submit built-in Iceberg update stats/metrics Spark jobs

Option Field Meanings

Option	Meaning	Used by
`--identifiers`	Comma-separated identifiers. Table format supports `catalog.schema.table` (or `schema.table` when default catalog is configured).	Most commands
`--strategy-name`	Policy name to evaluate, for example `iceberg_compaction_default`.	`submit-strategy-jobs`
`--dry-run`	Preview mode. Prints recommendations or job configs without submitting jobs.	`submit-strategy-jobs`, `submit-update-stats-job`
`--limit`	Maximum number of strategy jobs to process. Must be `> 0`.	`submit-strategy-jobs`
`--calculator-name`	Statistics/metrics calculator implementation name (for example `local-stats-calculator`).	`update-statistics`, `append-metrics`
`--statistics-payload`	Inline JSON Lines content as input. Mutually exclusive with `--file-path`.	`update-statistics`, `append-metrics`
`--file-path`	Path to JSON Lines input file. Mutually exclusive with `--statistics-payload`.	`update-statistics`, `append-metrics`
`--action-time`	Action timestamp in epoch seconds used as evaluation anchor.	`monitor-metrics`
`--range-seconds`	Time window (seconds) for monitor evaluation. Default is `86400` (24h).	`monitor-metrics`
`--partition-path`	Partition path JSON array, for example `'[{"dt":"2026-01-01"}]'`. Requires exactly one identifier.	`monitor-metrics`, `list-table-metrics`
`--update-mode`	Controls what built-in update job updates: `stats`, `metrics`, or `all` (default).	`submit-update-stats-job`
`--updater-options`	Flat JSON map passed to updater logic. For `stats`/`all`, include `gravitino_uri` and `metalake`.	`submit-update-stats-job`
`--spark-conf`	Flat JSON map of Spark and Iceberg catalog configs used by the job.	`submit-update-stats-job`

Global option:

--conf-path: Optional custom config file path. If omitted, CLI uses conf/gravitino-optimizer.conf.

Input Format for `local-stats-calculator`

local-stats-calculator reads JSON Lines (one JSON object per line).

Reserved Fields

stats-type: table, partition, or job
identifier: object identifier
partition-path: only for partition data, for example {"dt":"2026-01-01"}
timestamp: optional epoch seconds (record-level default timestamp for metric points)

All other fields are treated as metric or statistic values.

Supported Examples by Scope

Use JSON Lines (one JSON object per line). The following examples focus on table, partition, and job scopes with multiple metric/statistic fields:

{"stats-type":"table","identifier":"catalog.db.t1","timestamp":1735689600,"row_count":100}
{"stats-type":"table","identifier":"catalog.db.t1","row_count":100,"total_file_size":1048576}
{"stats-type":"table","identifier":"catalog.db.t1","timestamp":1735689660,"row_count":120,"file_count":24,"avg_file_size":10485.76}
{"stats-type":"partition","identifier":"catalog.db.t1","timestamp":1735689720,"partition-path":{"dt":"2026-01-01"},"row_count":20}
{"stats-type":"partition","identifier":"catalog.db.t1","partition-path":{"dt":"2026-01-01","region":"us"},"row_count":12,"file_count":3}
{"stats-type":"job","identifier":"job-1","timestamp":1735689800,"duration_ms":12500,"rewritten_files":18}

Identifier Rules

Table and partition records: catalog.schema.table
If gravitino.optimizer.gravitinoDefaultCatalog is set, schema.table is also accepted
Job records: parsed as a regular Gravitino NameIdentifier

CLI Workflow Examples

Batch Statistics Update

Calculate and persist table or partition statistics from JSONL input.

./bin/gravitino-optimizer.sh \
  --type update-statistics \
  --calculator-name local-stats-calculator \
  --file-path ./table-stats.jsonl

Batch Metrics Append

Calculate and append table or job metrics from JSONL input.

./bin/gravitino-optimizer.sh \
  --type append-metrics \
  --calculator-name local-stats-calculator \
  --file-path ./table-stats.jsonl

Dry-Run Strategy Submission

Preview recommendations without actually submitting jobs.

./bin/gravitino-optimizer.sh \
  --type submit-strategy-jobs \
  --identifiers rest_catalog.db.t1 \
  --strategy-name iceberg_compaction_default \
  --dry-run \
  --limit 10

Submit Strategy Jobs

Submit jobs for identifiers that match the given policy name.

./bin/gravitino-optimizer.sh \
  --type submit-strategy-jobs \
  --identifiers rest_catalog.db.t1 \
  --strategy-name iceberg_compaction_default \
  --limit 10

Monitor Metrics

Evaluate monitor rules around an action time.

./bin/gravitino-optimizer.sh \
  --type monitor-metrics \
  --identifiers catalog.db.sales \
  --action-time 1735689600 \
  --range-seconds 86400

Configure evaluator rules in gravitino-optimizer.conf:

gravitino.optimizer.monitor.gravitinoMetricsEvaluator.rules = table:row_count:avg:le,job:duration:latest:le

Rule format is scope:metricName:aggregation:comparison:

scope: table or job (table rules also apply to partition scope)
aggregation: max|min|avg|latest
comparison: lt|le|gt|ge|eq|ne

When metrics are produced by submit-update-stats-job --update-mode metrics, metric names are often custom-* (for example custom-data-file-mse). Use list-table-metrics first and configure rules with the exact metric names returned by your environment.

Submit Built-In Update Stats Jobs

Submit built-in Iceberg update stats/metrics Spark jobs directly.

./bin/gravitino-optimizer.sh \
  --type submit-update-stats-job \
  --identifiers rest_catalog.db.t1 \
  --update-mode all \
  --updater-options '{"gravitino_uri":"http://localhost:8090","metalake":"test"}' \
  --spark-conf '{"spark.sql.catalog.rest_catalog.type":"rest","spark.sql.catalog.rest_catalog.uri":"http://localhost:9001/iceberg","spark.hadoop.fs.defaultFS":"file:///"}'

Notes:

--identifiers supports catalog.schema.table or schema.table (when default catalog is configured).
--update-mode supports stats|metrics|all (default all).
For stats or all, --updater-options must include gravitino_uri and metalake.
If --updater-options includes external JDBC metrics settings (gravitino.optimizer.jdbcMetrics.*), ensure the JDBC driver JAR is available to Spark runtime classpath (for example via spark.jars in --spark-conf).
--spark-conf and --updater-options are flat JSON maps.

List Table Metrics

Query stored metrics at table scope.

./bin/gravitino-optimizer.sh \
  --type list-table-metrics \
  --identifiers catalog.db.sales

For partition scope, provide a partition path JSON array:

./bin/gravitino-optimizer.sh \
  --type list-table-metrics \
  --identifiers catalog.db.sales \
  --partition-path '[{"dt":"2026-01-01"}]'

List Job Metrics

Query stored metrics at job scope.

./bin/gravitino-optimizer.sh \
  --type list-job-metrics \
  --identifiers catalog.db.optimizer_job

Output Guide

SUMMARY: ...: summary for update-statistics and append-metrics
DRY-RUN: ...: recommendation preview without job submission
SUBMIT: ...: strategy job or built-in update-stats job submitted successfully
SUMMARY: submit-update-stats-job ...: summary for built-in update-stats submission
MetricsResult{...}: returned by list commands
EvaluationResult{...}: returned by monitor command

Examples:

SUMMARY: statistics totalRecords=3 tableRecords=2 partitionRecords=1 jobRecords=0
DRY-RUN: strategy=iceberg-data-compaction identifier=rest_catalog.db.t1 score=95 jobTemplate=builtin-iceberg-rewrite-data-files jobOptions={catalog_name=rest_catalog, table_identifier=db.t1}
SUBMIT: strategy=iceberg-data-compaction identifier=rest_catalog.db.t1 score=95 jobTemplate=builtin-iceberg-rewrite-data-files jobOptions={catalog_name=rest_catalog, table_identifier=db.t1} jobId=1f54c6d3-4e27-4cc8-bdfa-b05ecf59a4c2
DRY-RUN: identifier=rest_catalog.db.t1 jobTemplate=builtin-iceberg-update-stats jobConfig={catalog_name=rest_catalog, table_identifier=db.t1, update_mode=all, updater_options={"gravitino_uri":"http://localhost:8090","metalake":"test"}, spark_conf={"spark.master":"local[2]","spark.hadoop.fs.defaultFS":"file:///"}}
SUMMARY: submit-update-stats-job total=1 submitted=1 dryRun=false
MetricsResult{scopeType=TABLE, identifier=rest_catalog.db.t1, partitionPath=<table-or-job-scope>, metrics={row_count=[{timestamp=1735689600, value=100}]}}
EvaluationResult{scopeType=TABLE, identifier=rest_catalog.db.t1, partitionPath=<table-or-job-scope>, evaluation=true, evaluatorName=gravitino-metrics-evaluator, actionTimeSeconds=1735689600, rangeSeconds=86400, beforeMetrics={row_count=[MetricSample{timestampSeconds=1735686000, value=120}]}, afterMetrics={row_count=[MetricSample{timestampSeconds=1735689600, value=100}]}}

Troubleshooting

Failures fall into three groups, matching where they occur in the workflow. Command and argument errors surface immediately. Evaluation problems produce no output rather than an error, which is what makes them confusing. Execution failures happen inside Spark, so the real message is in the staging log rather than the API response.

Staging logs live under /tmp/gravitino/jobs/staging/{metalake}/{job_template_name}/{job_id}/, controlled by gravitino.job.stagingDir. Read error.log for failures and output.log for results.

Command and Argument Errors

These come back from the CLI immediately and name the problem.

Invalid --type: command names are kebab-case. Use update-statistics, not update_statistics.

--statistics-payload and --file-path cannot be used together: local-stats-calculator takes exactly one input source.

requires one of --statistics-payload or --file-path: the same rule from the other side. With --calculator-name local-stats-calculator, one of the two is mandatory.

--partition-path must be a JSON array: even for a single partition, pass an array:

[{"dt":"2026-01-01"}]

Specified optimizer config file does not exist: check the --conf-path value and the file's permissions.

No StrategyHandler class configured for strategy type ...: the strategy handler mapping is missing from the CLI configuration:

gravitino.optimizer.strategyHandler.iceberg-data-compaction.className = org.apache.gravitino.maintenance.optimizer.recommender.handler.compaction.CompactionStrategyHandler

The packaged default configuration already contains this, so seeing it usually means a hand-written config file.

Evaluation Produces Nothing

These are the hard ones, because success and "the policy decided not to act" look identical.

No identifiers matched strategy name ...: --strategy-name takes the policy name, for example iceberg_compaction_default. It does not take the policy type system_iceberg_compaction or the strategy type iceberg-data-compaction, despite being called strategy name.

A dry run prints no DRY-RUN or SUBMIT lines: the trigger conditions were not met. For compaction, check that custom-data-file-mse and custom-delete-file-number in the table's statistics are large enough to satisfy the policy rules. A table with too few small files is the usual cause, and the fix is more data rather than more configuration.

monitor-metrics returns evaluation=false unexpectedly: check the rule names and the sample window together:

Query the current metrics with list-table-metrics, adding --partition-path for partition scope.
Use the exact metric names your environment returns in gravitino.optimizer.monitor.gravitinoMetricsEvaluator.rules. Names that look close enough are not.
Make sure --action-time falls inside a range where both a before and an after sample exist.

Job Execution Failures

Status stays queued or started for a long time: REST status is polled, not pushed, and gravitino.job.statusPullIntervalInMs defaults to five minutes. Lower it to 10000 and restart the server for local work. If the status is genuinely stuck rather than lagging, read error.log in the staging directory.

Spark fails with hdfs://localhost:9000 or other filesystem errors: Spark is defaulting to HDFS on a machine that has none:

spark.hadoop.fs.defaultFS=file:///

submit-update-stats-job fails with JDBC metrics errors: when --updater-options includes gravitino.optimizer.jdbcMetrics.*, the JDBC driver has to be on the Spark runtime classpath. ClassNotFoundException and No suitable driver both mean the same thing:

{
  "spark.jars": "/path/to/postgresql-42.7.4.jar"
}

Rewrite fails on a multi-level partition: rewriting a table partitioned by an identity transform combined with a time transform, such as PARTITIONED BY (p, days(ts)), fails with:

Cannot translate Spark expression ... day(cast(ts as date)) ... to data source filter

Confirm it by checking the job run at /api/metalakes/{metalake}/jobs/runs/{job_id} and reading error.log under builtin-iceberg-rewrite-data-files. The only workaround is to compact identity-partitioned tables and leave the rest alone.

Observed behavior by partitioning:

Partitioning	Rewrite
`p`, `p, c2`	Works
`p, years(ts)`, `p, months(ts)`, `p, days(ts)`, `p, hours(ts)`	Fails
`p, truncate(1, c2)`, `p, bucket(8, id)`	Fails

Table Maintenance for the concepts, configuration, and the walkthrough
Templates for the built-in job templates
Iceberg Compaction Policy for tuning the built-in strategy
Jobs for job status and output

Overview​

Command Quick Reference​

Option Field Meanings​

Input Format for local-stats-calculator​

Reserved Fields​

Supported Examples by Scope​

Identifier Rules​

CLI Workflow Examples​

Batch Statistics Update​

Batch Metrics Append​

Dry-Run Strategy Submission​

Submit Strategy Jobs​

Monitor Metrics​

Submit Built-In Update Stats Jobs​

List Table Metrics​

List Job Metrics​

Output Guide​

Troubleshooting​

Command and Argument Errors​

Evaluation Produces Nothing​

Job Execution Failures​

Related​