Connectors on Kubernetes

The Gravitino Helm chart installs the Gravitino server. It does not install anything into Trino, Spark, or Flink. Each connector is a separate build artifact that has to reach the engine's own container, and the way it gets there differs by engine because the three connectors ship in three different shapes.

This page covers getting each connector into an engine running on Kubernetes, and pointing it at the Gravitino service.

Quick Start

Spark and Flink take a JAR from Maven Central. Spark can resolve it at submit time, and Flink needs it in the image or fetched into the lib directory before startup.

Trino takes a directory, not a JAR. The Trino connector is distributed as a tarball on GitHub Releases that unpacks into a plugin directory, so it has to be present in the image or placed there before Trino starts.

All three need the in-cluster address of the Gravitino service, which the chart publishes as gravitino on port 8090 by default.

http://gravitino.{namespace}.svc.cluster.local:8090

Confirm the name in your cluster, since the chart takes it from a values key rather than deriving it from the release name.

kubectl get svc -n {namespace}

How Each Connector Ships

Engine	Artifact	Published to	Placement
Spark	One runtime JAR	Maven Central	Spark classpath
Flink	One runtime JAR	Maven Central	Flink lib directory
Trino	An unpacked directory	GitHub Releases	Trino plugin directory

The Trino row is the one that makes this page necessary. Because the Trino connector is not published to Maven Central and is a directory rather than a JAR, the resolution mechanisms Spark and Flink users reach for are unavailable, and every approach involves getting a tarball onto a filesystem before the engine starts.

Spark

The Spark connector is published as org.apache.gravitino:gravitino-spark-connector-runtime-{spark_version} for Spark 3.3, 3.4, and 3.5. Because it resolves from Maven Central, no image change is needed.

Add the coordinate to the submitted application rather than to the image.

spark.jars.packages: org.apache.gravitino:gravitino-spark-connector-runtime-3.5:{gravitino_version}
spark.plugins: org.apache.gravitino.spark.connector.plugin.GravitinoSparkPlugin
spark.sql.gravitino.uri: http://gravitino.{namespace}.svc.cluster.local:8090
spark.sql.gravitino.metalake: {metalake_name}

Resolution happens at submit time, so a cluster without egress to Maven Central needs either an internal mirror configured through the usual Spark repository settings or the JAR baked into the Spark image.

Using an Iceberg catalog also requires the Iceberg runtime JAR on the classpath, which is a separate artifact and is subject to the same consideration.

Flink

The Flink connector is published per Flink minor version, and the runtime JAR has to be present before the job manager starts rather than being resolved at submit time.

Flink version	Runtime artifact
1.18	gravitino-flink-connector-runtime-1.18_2.12-{version}.jar
1.19	gravitino-flink-connector-runtime-1.19_2.12-{version}.jar
1.20	gravitino-flink-connector-runtime-1.20_2.12-{version}.jar

Only Scala 2.12 is supported, and runtime JARs from different Flink minor versions must not be mixed in one deployment.

The straightforward approach is a derived image.

FROM flink:1.20
ADD https://repo1.maven.org/maven2/org/apache/gravitino/gravitino-flink-connector-runtime-1.20_2.12/{version}/gravitino-flink-connector-runtime-1.20_2.12-{version}.jar /opt/flink/lib/

An init container writing into a volume shared with the Flink container works as well, and avoids rebuilding an image for a version bump at the cost of a network fetch on every pod start.

Point the connector at Gravitino through the Flink configuration.

table.catalog-store.kind: gravitino
table.catalog-store.gravitino.gravitino.uri: http://gravitino.{namespace}.svc.cluster.local:8090
table.catalog-store.gravitino.gravitino.metalake: {metalake_name}

Trino

Three things have to be true before Trino loads the connector. The plugin directory has to contain an unpacked connector directory named gravitino, the coordinator has to run with dynamic catalog management, and a catalog properties file has to point at the Gravitino server.

Choosing the Package

Gravitino builds the Trino connector against six Trino version segments, and the package has to match the Trino server.

Trino server version	Package segment
435 to 439	trino-connector-435-439
440 to 445	trino-connector-440-445
446 to 451	trino-connector-446-451
452 to 468	trino-connector-452-468
469 to 472	trino-connector-469-472
473 to 478	trino-connector-473-478

The release archive is named gravitino-trino-connector-{segment}-{version}.tar.gz and is published on the project's GitHub releases rather than to a package repository.

Building an Image

Baking the connector into an image is the approach that survives restarts, rescheduling, and scaling without any further mechanism, and it is what a production deployment should use.

FROM trinodb/trino:469

ARG GRAVITINO_VERSION
ADD https://github.com/apache/gravitino/releases/download/${GRAVITINO_VERSION}/gravitino-trino-connector-469-472-${GRAVITINO_VERSION}.tar.gz /tmp/

USER root
RUN tar -xzf /tmp/gravitino-trino-connector-469-472-${GRAVITINO_VERSION}.tar.gz -C /tmp \
    && mv /tmp/gravitino-trino-connector-469-472-${GRAVITINO_VERSION} /usr/lib/trino/plugin/gravitino \
    && rm /tmp/gravitino-trino-connector-469-472-${GRAVITINO_VERSION}.tar.gz
USER trino

The plugin path differs between Trino images and distributions, so confirm it rather than assuming. Renaming the unpacked directory to gravitino is required, not cosmetic.

Using an Init Container

An init container that fetches and unpacks the tarball into a volume shared with the Trino container avoids maintaining an image, at the cost of a download on every pod start and a dependency on reaching GitHub from inside the cluster.

initContainers:
  - name: gravitino-plugin
    image: curlimages/curl:latest
    command:
      - sh
      - -c
      - |
        curl -sSL -o /tmp/c.tar.gz \
          https://github.com/apache/gravitino/releases/download/{version}/gravitino-trino-connector-469-472-{version}.tar.gz
        mkdir -p /plugins/gravitino
        tar -xzf /tmp/c.tar.gz -C /plugins/gravitino --strip-components=1
    volumeMounts:
      - name: gravitino-plugin
        mountPath: /plugins

Mount the same volume into the Trino container at its plugin path. Whether your Trino chart exposes init containers and additional volumes varies by chart, so check its values before committing to this approach.

Configuring Trino

The coordinator needs dynamic catalog management, without which the catalogs Gravitino exposes never appear.

catalog.management=dynamic

The catalog properties file names the connector and the Gravitino server.

connector.name=gravitino
gravitino.uri=http://gravitino.{namespace}.svc.cluster.local:8090
gravitino.metalake={metalake_name}

Most Trino charts render both of these from values rather than requiring a mounted file.

Verifying

For Trino, list the catalogs and confirm that gravitino appears alongside the catalogs your metalake exposes.

kubectl exec -it {trino_pod} -n {namespace} -- trino --execute "SHOW CATALOGS"

A missing gravitino catalog usually means the plugin directory name is wrong or the plugin is absent. Catalogs from the metalake missing while gravitino is present usually means dynamic catalog management is off or the server address is unreachable from the pod.

Copying Into a Running Container

Some existing material demonstrates installing the Trino connector by copying it into a running container. That works for a local demonstration and does not work on Kubernetes, because the container filesystem is discarded when the pod restarts, and a scaled or rescheduled pod starts without it. Use an image or an init container instead.