Jobs

A job is one run of a job template. The UI lists the metalake's jobs with the template each ran from and its status, and a job can be started from a template or cancelled from there.

Quick start

1. Run a template. jobConf supplies values for the template's parameters.

   Run a job template

   curl -sS -X POST "https://{gravitino_host}/api/metalakes/{metalake}/jobs/runs" \
     -H "Accept: application/vnd.gravitino.v1+json" \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $TOKEN" \
     -d '{"jobTemplateName": "{template}", "jobConf": {"{parameter}": "{value}"}}'

   The response carries the job's id.

2. Check its status.

   Get the job status

   curl -sS "https://{gravitino_host}/api/metalakes/{metalake}/jobs/runs/{job_id}" \
     -H "Accept: application/vnd.gravitino.v1+json" \
     -H "Authorization: Bearer $TOKEN"

3. Read its output. Add includeOutput=true to return the job's captured stdout and stderr with it, so a failure can be diagnosed without access to the server's staging directory.

   Get the job with its output

   curl -sS "https://{gravitino_host}/api/metalakes/{metalake}/jobs/runs/{job_id}?includeOutput=true" \
     -H "Accept: application/vnd.gravitino.v1+json" \
     -H "Authorization: Bearer $TOKEN"

Running a job in the UI

1. Open Automate → Jobs and click Run job.
2. Choose the Template name. The template's executable, arguments, and Spark configuration appear under Execution, with a placeholder for each value the run must supply.
3. Fill in the required values, marked with an asterisk, and any optional ones.
4. Click Run job.

The Jobs list shows each run with its job ID, template, status, who started it, when, and how long it ran.

Job status

Status	Meaning
QUEUED	Waiting to be executed
STARTED	Currently executing
SUCCEEDED	Finished successfully
FAILED	Finished with an error
CANCELLING	Cancellation requested, not yet complete
CANCELLED	Cancellation complete

Status is polled from the executor rather than pushed, so it can lag the real process by up to gravitino.job.statusPullIntervalInMs. A job that reads as running for a few minutes after it finished is lagging, not stuck. See Automation.

Cancelling a job

Cancellation is a request rather than an instant, which is why CANCELLING and CANCELLED are separate. A job that finishes before the cancellation lands stays in its finished state.

Cancel a job

curl -sS -X POST "https://{gravitino_host}/api/metalakes/{metalake}/jobs/runs/{job_id}" \
  -H "Accept: application/vnd.gravitino.v1+json" \
  -H "Authorization: Bearer $TOKEN"

Endpoints

Paths are relative to https://{gravitino_host}/api/metalakes/{metalake}.

Operation	Method	Path
Run a template	POST	/jobs/runs
List jobs	GET	/jobs/runs
List jobs for one template	GET	/jobs/runs?jobTemplateName={template}
Get a job	GET	/jobs/runs/{job_id}
Cancel a job	POST	/jobs/runs/{job_id}

Running a job takes RUN_JOB and USE_JOB_TEMPLATE for its template. See Automation.