Post run event

POST 
/lineage

Records an OpenLineage run event that reports a change in a job run's state, and returns 201 with an empty body. When authorization is enabled, every input and output dataset must name a Gravitino object in the metalake given by job.namespace that the caller can access.

Sign in with a bearer token or a local user's password; see Authentication.

Request

application/json

Body

eventTime date-timerequired

the time the event occurred at

producer urirequired

URI identifying the producer of this metadata. For example this could be a git url with a given tag or sha

schemaURL urirequired

The JSON Pointer (https://tools.ietf.org/html/rfc6901) URL to the corresponding version of the schema definition for this RunEvent

eventType string

Possible values: [START, RUNNING, COMPLETE, ABORT, FAIL, OTHER]

the current transition of the run state. It is required to issue 1 START event and 1 of [ COMPLETE, ABORT, FAIL ] event per run. Additional events with OTHER eventType can be added to the same run. For example to send additional metadata after the run is complete

run

object

required

runId uuidrequired

The globally unique ID of the run associated with the job.

facets

object

The run facets.

anyOf

MOD1

property name*

RunFacet

A Run Facet

_producer urirequired

URI identifying the producer of this metadata. For example this could be a git url with a given tag or sha

_schemaURL urirequired

The JSON Pointer (https://tools.ietf.org/html/rfc6901) URL to the corresponding version of the schema definition for this facet

property name* any

A Run Facet

job

object

required

namespace stringrequired

The namespace containing that job. With authorization enabled, it is the metalake name, and the caller must be a member of that metalake; an event with no input or output datasets needs nothing more. With authorization disabled, any OpenLineage namespace is accepted.

name stringrequired

The unique name for that job within that namespace

facets

object

The job facets.

anyOf

MOD1

property name*

JobFacet

A Job Facet

_producer urirequired

URI identifying the producer of this metadata. For example this could be a git url with a given tag or sha

_schemaURL urirequired

The JSON Pointer (https://tools.ietf.org/html/rfc6901) URL to the corresponding version of the schema definition for this facet

_deleted boolean

set to true to delete a facet

property name* any

A Job Facet

inputs

object[]

The set of input datasets.

Array [

namespace stringrequired

The namespace containing that dataset; with authorization enabled, the metalake name

name stringrequired

The unique name for that dataset within that namespace; with authorization enabled, its Gravitino full name

facets

object

The facets for this dataset. With authorization enabled, an optional datasetType facet, whose datasetType field is one of TABLE, VIEW, FILE, FILESET, MODEL or TOPIC (case-insensitive; FILE is treated as FILESET), tells the server which kind of Gravitino object the dataset name refers to. Without the facet the dataset is treated as a table, and any other value is rejected.

anyOf

MOD1

property name*

DatasetFacet

A Dataset Facet

_producer urirequired

URI identifying the producer of this metadata. For example this could be a git url with a given tag or sha

_schemaURL urirequired

The JSON Pointer (https://tools.ietf.org/html/rfc6901) URL to the corresponding version of the schema definition for this facet

_deleted boolean

set to true to delete a facet

property name* any

A Dataset Facet

inputFacets

object

The input facets for this dataset.

anyOf

MOD1

property name*

InputDatasetFacet

An Input Dataset Facet

_producer urirequired

URI identifying the producer of this metadata. For example this could be a git url with a given tag or sha

_schemaURL urirequired

The JSON Pointer (https://tools.ietf.org/html/rfc6901) URL to the corresponding version of the schema definition for this facet

property name* any

An Input Dataset Facet

]

outputs

object[]

The set of output datasets.

Array [

namespace stringrequired

The namespace containing that dataset; with authorization enabled, the metalake name

name stringrequired

The unique name for that dataset within that namespace; with authorization enabled, its Gravitino full name

facets

object

The facets for this dataset. With authorization enabled, an optional datasetType facet, whose datasetType field is one of TABLE, VIEW, FILE, FILESET, MODEL or TOPIC (case-insensitive; FILE is treated as FILESET), tells the server which kind of Gravitino object the dataset name refers to. Without the facet the dataset is treated as a table, and any other value is rejected.

anyOf

MOD1

property name*

DatasetFacet

A Dataset Facet

_producer urirequired

URI identifying the producer of this metadata. For example this could be a git url with a given tag or sha

_schemaURL urirequired

The JSON Pointer (https://tools.ietf.org/html/rfc6901) URL to the corresponding version of the schema definition for this facet

_deleted boolean

set to true to delete a facet

property name* any

A Dataset Facet

outputFacets

object

The output facets for this dataset

anyOf

MOD1

property name*

OutputDatasetFacet

An Output Dataset Facet

_producer urirequired

URI identifying the producer of this metadata. For example this could be a git url with a given tag or sha

_schemaURL urirequired

The JSON Pointer (https://tools.ietf.org/html/rfc6901) URL to the corresponding version of the schema definition for this facet

property name* any

An Output Dataset Facet

]

Responses

201

The event was accepted and queued for the configured lineage sinks. The response has no body.

400

The request is malformed or fails validation, for example because of invalid JSON, a missing required field or an invalid parameter value. The body is normally an error model, although the server's middleware can return plain text instead.

application/vnd.gravitino.v1+json

Schema

Schema

code integerrequired

Possible values: >= 1000 and <= 1100

HTTP response code

type stringrequired

Internal type definition of the error

message stringrequired

A human-readable message

stack string[]deprecated

Deprecated diagnostic stack trace. Enterprise omits it by default because it can expose internal implementation details. Operators can opt in for controlled diagnostics with gravitino.server.webserver.includeErrorStackTrace=true. See OWASP REST Security: Error handling and CWE-209.

Example (from schema)

{
  "code": 1002,
  "type": "string",
  "message": "string"
}

Example

{
  "code": 1003,
  "type": "BadRequestException",
  "message": "Malformed request"
}

403

The caller is not a member of the metalake named by job.namespace, or cannot access at least one of the event's input or output datasets.

application/vnd.gravitino.v1+json

Schema

Schema

code integerrequired

Possible values: >= 1000 and <= 1100

HTTP response code

type stringrequired

Internal type definition of the error

message stringrequired

A human-readable message

stack string[]deprecated

Deprecated diagnostic stack trace. Enterprise omits it by default because it can expose internal implementation details. Operators can opt in for controlled diagnostics with gravitino.server.webserver.includeErrorStackTrace=true. See OWASP REST Security: Error handling and CWE-209.

Example (from schema)

{
  "code": 1002,
  "type": "string",
  "message": "string"
}

429

The lineage sink queue is full, so the event was not recorded and should be retried later. The response has no body.

5xx

The server hit an internal error that the client usually cannot fix by changing the request.

application/vnd.gravitino.v1+json

Schema

Schema

code integerrequired

Possible values: >= 1000 and <= 1100

HTTP response code

type stringrequired

Internal type definition of the error

message stringrequired

A human-readable message

stack string[]deprecated

Deprecated diagnostic stack trace. Enterprise omits it by default because it can expose internal implementation details. Operators can opt in for controlled diagnostics with gravitino.server.webserver.includeErrorStackTrace=true. See OWASP REST Security: Error handling and CWE-209.

Example (from schema)

{
  "code": 1002,
  "type": "string",
  "message": "string"
}

Example

{
  "code": 1002,
  "type": "RuntimeException",
  "message": "Internal Server Error",
  "stack": [
    "java.lang.RuntimeException: Internal Server Error"
  ]
}

Loading...