Automation
A job is a piece of work Gravitino runs on your behalf, and a job template is the reusable definition it runs from. Registering a template once and running it many times is the shape, so the command, its arguments, and its environment live in the catalog rather than in whoever launched it last.
Jobs are how catalog-driven work gets done. Table maintenance uses them, and anything else that should run against metadata Gravitino already knows about can use the same mechanism.
The UI summarizes the metalake's jobs at a glance: how many are running, how many failed or completed in the last day, and the most recent runs with their templates and status.
Quick Start
1. Register a template, or use a built-in one. A template names the kind of job, the command or application to run, and the parameters it takes. See Templates.
2. Run it. Running a template creates a job, with values for the template's parameters. See Jobs.
3. Watch it. A job reports its status as it moves through the queue, and finished jobs keep their outcome and output.
Templates and Jobs
A template is the definition and a job is one execution of it. Templates are named and reusable; jobs are individual and carry a status. Deleting a template does not affect jobs already run from it.
Permissions
| Privilege | Grantable On | What It Allows |
|---|---|---|
REGISTER_JOB_TEMPLATE | Metalake | Registering job templates |
USE_JOB_TEMPLATE | Metalake, or one template | Reading and using a job template |
RUN_JOB | Metalake | Running a job |
Running a job takes both RUN_JOB and USE_JOB_TEMPLATE for the template it runs from. Altering and
deleting a job template are reserved for the metalake owner and the template owner.
Server Configuration
| Configuration Item | Description | Default Value |
|---|---|---|
gravitino.job.executor | Executor that runs jobs | local |
gravitino.job.stagingDir | Directory holding each job's staging files and logs | /tmp/gravitino/jobs/staging |
gravitino.job.statusPullIntervalInMs | Interval between job status polls | 300000 (5 minutes) |
gravitino.jobExecutor.local.sparkHome | Spark installation the local executor submits Spark jobs with | (none) |
Job status is polled rather than pushed, so a job's reported status can lag the real process by a full interval. At the default, a job that finished seconds ago can still read as running for five minutes, which is worth lowering for interactive work.
API
Templates and jobs can be registered, listed, run, and cancelled over REST and through the Java and Python clients. Endpoints, payload shapes, and worked examples are in Manage Jobs.