Server Lineage

Overview

The server receives OpenLineage run events, passes each through a processor, and writes it to one or more sinks. Each stage is pluggable, so events can be transformed or sent to systems beyond the bundled lineage store.

Quick Start

1. Send an event. The HTTP source accepts events that follow the OpenLineage API specification.

curl -sS -X POST "https://{gravitino_host}/api/lineage" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
  "eventType": "START",
  "eventTime": "2026-01-01T00:00:00.000Z",
  "run": {"runId": "0176a8c2-fe01-7439-87e6-56a1a1b4029f"},
  "job": {"namespace": "{metalake}", "name": "{job_name}"},
  "inputs": [{"namespace": "{metalake}", "name": "{catalog}.{schema}.{table}"}],
  "producer": "https://github.com/OpenLineage/OpenLineage/blob/v1-0-0/client",
  "schemaURL": "https://openlineage.io/spec/1-0-5/OpenLineage.json#/definitions/RunEvent"
}'

2. Confirm it arrived. The dataset appears in the lineage view under the metalake's namespace.

Configuration

Configuration Item	Description	Default Value
gravitino.lineage.source	Name of the event source	http
gravitino.lineage.{source}.sourceClass	Class implementing org.apache.gravitino.lineage.source.LineageSource	(none)
gravitino.lineage.processorClass	Class implementing org.apache.gravitino.lineage.processor.LineageProcessor	org.apache.gravitino.lineage.processor.NoopProcessor
gravitino.lineage.sinks	Comma-separated sink names	log
gravitino.lineage.{sink}.sinkClass	Class implementing org.apache.gravitino.lineage.sink.LineageSink	(none)
gravitino.lineage.queueCapacity	Total queue capacity, divided evenly across sinks since each has its own queue	10000

The default processor passes events through unchanged.

Sinks

Log Sink

Writes each event to gravitino_lineage.log, whose location and rotation are set in the server's Log4j2 configuration. Useful for checking that events arrive before a lineage store is connected.

HTTP Sink

Sends each event to a server that follows the OpenLineage REST specification, which is how events reach the bundled lineage store.

Configuration Item	Description	Default Value
gravitino.lineage.http.sinkClass	org.apache.gravitino.lineage.sink.LineageHttpSink	(none)
gravitino.lineage.http.url	Base URL of the receiving server	(none)
gravitino.lineage.http.authType	apiKey or none	none
gravitino.lineage.http.apiKey	API key, when authType is apiKey	(none)

Set gravitino.lineage.sinks to include http for the sink to run.

When Sinks Fall Behind

Each sink reads from a queue. When a queue passes 90 percent of its capacity, the server stops accepting events and the HTTP source answers 429 Too Many Requests, so a producer can back off and retry rather than lose events silently. A producer that does not retry on 429 drops those events.