Gravitino server configuration
Configure → Settings in the UI shows the server configuration the chart wrote into the server's
gravitino.conf ConfigMap. The page is read only and says so ("Managed by Helm and read only"):
every change goes through my-values.yaml, as described below.
Keys are grouped as follows. Each group shows its key count, Expand all and Collapse all open and close every group, the search box filters the listed keys by name or value, and each key has a copy button.
| Group | Keys |
|---|---|
| Server | gravitino.server.*, except gravitino.server.webserver.* |
| Webserver | gravitino.server.webserver.* |
| Iceberg REST | gravitino.iceberg-rest.* |
| Lance REST | gravitino.lance-rest.* |
| Metadata database | gravitino.entity.store and gravitino.entity.store.* |
| Security | gravitino.authorization.*, gravitino.authenticators, gravitino.authenticator.*, gravitino.scim.* |
| Events | gravitino.eventListener.* |
| Web UI | The UI's own session timeouts, read from the UI deployment rather than the ConfigMap |
A key ending in password, secret, token, keytab, or signkey shows Write-only, and its
value never reaches the browser. Apart from Web UI, the page shows only keys present in the
ConfigMap that fall in one of these groups. A property left at its server default does not appear,
and neither does a key outside the groups, such as gravitino.audit.*, gravitino.cache.*, or
gravitino.job.*.
The server reads gravitino.conf once, at startup. Properties the chart does not write take the
server's default. Server properties come first. Catalog properties, which configure an
individual catalog rather than the server, are covered further down. Properties for the auxiliary
services live with those services: see Iceberg REST service and
Security.
Quick start
The example raises the largest HTTP request header the server accepts, a common change when long OAuth tokens arrive in headers. Every server property follows the same three steps.
-
Find the property and its chart value. The tables below name the chart value for each property that has one.
gravitino.server.webserver.requestHeaderSizeis set bygravitino.webserver.requestHeaderSize. -
Set it in
my-values.yamland apply it. Rerun thehelm upgrade --installcommand from Install with the same values file. The server pod restarts with the new value.my-values.yamlgravitino:
webserver:
requestHeaderSize: 262144A property with no chart value goes under
gravitino.additionalConfigItemsas a quoted string instead; see Set other server properties. -
Check it. Open Configure → Settings, expand Webserver, and search for
requestHeaderSize.
For the settings a production deployment changes, see the Production checklist.
Entity cache statistics
gravitino.cache.enableStats logs hit count, miss count, and load failures every five minutes,
which is what makes a cache problem visible in production. The chart already sets
gravitino.cache.enabled to true and gravitino.cache.implementation to caffeine.
gravitino:
cache:
enableStats: true
Raise gravitino.cache.lockSegments, which has no chart value, above its default of 16 under
gravitino.additionalConfigItems if the server runs hot enough for cache lock contention to show
up in profiles.
Configuration files and environment
On Kubernetes, nothing is configured by editing files in the pod. The chart writes the server's
configuration files from the values file into a ConfigMap, and the server pod copies them into
/opt/gravitino/conf each time it starts, so an edit made inside a running pod is gone after the
next restart.
| File | Holds | Set through |
|---|---|---|
gravitino.conf | Server properties | Chart values, and gravitino.additionalConfigItems for keys with no value of their own; see Set other server properties |
log4j2.properties | Logging | gravitino.log4j2Properties and gravitino.additionalLog4j2Properties; see Server log |
gravitino-env.sh | Defaults that come with the image | Not used for configuration |
Environment variables for the server go in gravitino.env, a list of name and value entries. The
start script reads these:
| Variable | Effect |
|---|---|
GRAVITINO_MEM | JVM memory flags: heap and Metaspace. The chart sets it |
JAVA_OPTS | Further JVM flags, added to the ones the start script sets |
GRAVITINO_DEBUG_OPTS | JVM flags for attaching a debugger |
HADOOP_USER_NAME | The user the server acts as on HDFS; see Apache Hadoop access |
GOOGLE_APPLICATION_CREDENTIALS | Path to a Google service account key; see Google Cloud credentials |
gravitino.env replaces the chart's list rather than adding to it, so keep the GRAVITINO_MEM
entry when you add another. The UI has its own list, enterpriseWeb.env; see
Session timeout.
Server configuration
Set a property in this section through its dedicated chart value when one exists, and otherwise
under gravitino.additionalConfigItems in my-values.yaml; see
Set other server properties. Each section names the chart
values for its properties, and a property it does not name has none. A change takes effect when the
Helm upgrade restarts the server pod.
Where the chart writes a value of its own, the Default value column shows the chart's value.
Otherwise it shows the server's default. (empty) means the property exists with an empty string
or list; (none) means it has no default at all.
Requests
HTTP server
| Configuration item | Description | Default value |
|---|---|---|
gravitino.server.webserver.host | The address the server binds to. | 0.0.0.0 |
gravitino.server.webserver.httpPort | The port the server listens on. | 8090 |
gravitino.server.webserver.minThreads | Minimum threads in the Jetty thread pool. Values below 8 are raised to 8. | 24 |
gravitino.server.webserver.maxThreads | Maximum threads in the Jetty thread pool. Values below 8 are raised to 8, and the value must be at least minThreads. | 200 |
gravitino.server.webserver.threadPoolWorkQueueSize | Size of the Jetty thread pool work queue. | 100 |
gravitino.server.webserver.idleTimeout | Timeout in milliseconds for idle connections. | 30000 |
gravitino.server.webserver.stopTimeout | Time in milliseconds Jetty waits for a graceful shutdown. See org.eclipse.jetty.server.Server#setStopTimeout. | 30000 |
gravitino.server.shutdown.timeout | Time in milliseconds for the Gravitino server itself to shut down gracefully. | 3000 |
gravitino.server.webserver.requestHeaderSize | Maximum size in bytes of an HTTP request header. | 131072 |
gravitino.server.webserver.responseHeaderSize | Maximum size in bytes of an HTTP response header. | 131072 |
gravitino.server.webserver.includeErrorStackTrace | Whether HTTP error responses carry the server-side stack trace. Stack traces expose implementation details, so turn this on only for controlled diagnostics. | false |
gravitino.server.webserver.customFilters | Comma-separated list of servlet filter class names to apply to the API. | (empty) |
gravitino.server.rest.extensionPackages | Comma-separated list of packages to scan for additional REST resources. | Three com.datastrato packages; see below |
gravitino.server.visibleConfigs | Comma-separated list of extra properties to expose on the unauthenticated GET /configs endpoint, on top of the fixed set it always returns. Additive, so each entry widens what is public. | gravitino.datastrato.custom.marquez.enable |
Chart values: each gravitino.server.webserver.{key} row is set by gravitino.webserver.{key},
gravitino.server.shutdown.timeout by gravitino.server.shutdownTimeout,
gravitino.server.rest.extensionPackages by the chart value of the same name, and
gravitino.server.visibleConfigs by gravitino.visibleConfigs, with the published values in
gravitino.visibleConfigsItems.
The chart's gravitino.server.rest.extensionPackages is
com.datastrato.gravitino.server.web.rest,com.datastrato.gravitino.search.rest,com.datastrato.gravitino.metrics.rest.
It appends com.datastrato.gravitino.scim.web.rest.feature when SCIM is on and
org.apache.gravitino.idp.web.rest.feature when basic is among the authenticators. Setting the
value yourself replaces the three com.datastrato packages, so list them along with your own. The
chart also appends gravitino.datastrato.custom.oidc.enable to visibleConfigs when
gravitino.browserSso is set.
Filters named in customFilters must be standard javax.servlet filters. Pass parameters to a
filter with properties of the form
gravitino.server.webserver.{filter_class_name}.param.{param_name} = {value}.
GET /configs backs the UI, so it answers without authentication. It returns
gravitino.authenticators, gravitino.authorization.enable, and gravitino.schema.separator,
the OAuth authenticator settings when oauth is among the authenticators, and each key listed in
visibleConfigs that has a value. Treat anything you add through visibleConfigs as public.
Clients reach the server over HTTPS through the ingress controller, which holds the certificates; see HTTPS. The CORS filter and its allowed origins, methods, and headers, needed when a browser client runs on a different origin than the server, are in CORS.
Schema names
Catalogs that support hierarchical schemas expose the hierarchy as a single delimited name at
the API boundary. Internally the levels are stored with ASCII-1 as the physical separator, so
the configured separator is an external representation only, and it may be neither blank, nor
., nor ASCII-1.
| Configuration item | Description | Default value |
|---|---|---|
gravitino.schema.separator | Separator representing a multi-level schema name at the API boundary, as in A:B:C. See Hierarchical schema. | : |
Health check endpoints
Gravitino exposes three health endpoints following MicroProfile Health semantics. All of them are exempt from authentication, so Kubernetes probes, load balancers, and traffic managers reach them without credentials.
| Endpoint | Root alias | Description | HTTP status |
|---|---|---|---|
GET /api/health/live | GET /health/live | Liveness. Returns 200 as long as an HTTP server thread can respond. Use it to decide whether to restart a pod. | 200 or 503 |
GET /api/health/ready | GET /health/ready | Readiness. Returns 200 when the metadata database answers within the probe timeout, 503 when it is unavailable or slow. Use it to route traffic. | 200 or 503 |
GET /api/health | GET /health | Aggregate. Returns 200 when both of the above pass. Also aliased as GET /health.html. | 200 or 503 |
| Configuration item | Description | Default value |
|---|---|---|
gravitino.server.health.entityStore.probeTimeoutMs | Timeout in milliseconds for the metadata database probe behind /ready. | 2000 |
Chart value: gravitino.server.health.entityStoreProbeTimeoutMs.
Every endpoint returns the same JSON shape, but not the same checks. code is always 0,
status is up or down, and checks carries one entry per component probed. /live reports
httpServer alone, /ready reports entityStore, and the aggregate endpoint reports both.
/ready and the aggregate also report the optional search check for the search backend. An
optional check is reported but never changes the status, so a search backend that is down leaves
the server in service:
{
"code": 0,
"status": "down",
"checks": [
{ "name": "httpServer", "status": "up", "details": {} },
{ "name": "entityStore", "status": "down", "details": { "reason": "timeout" } },
{ "name": "search", "status": "up", "details": {} }
]
}
A failing entityStore check reports timeout, interrupted, probe-rejected,
entity store not initialized, or the simple class name of an unexpected exception.
Once the server has recorded an OutOfMemoryError, every health endpoint, /live included,
returns 503 with a single jvm check whose reason is OutOfMemoryError; restart required.
JVM memory
The chart sets the JVM flags through the GRAVITINO_MEM entry in gravitino.env. Its default:
gravitino:
env:
- name: GRAVITINO_MEM
value: "-XX:MaxRAMPercentage=50 -XX:MaxMetaspaceSize=1g -XX:+ExitOnOutOfMemoryError"
The heap is half of the container's memory limit, so it grows with
gravitino.resources.limits.memory, which defaults to 4Gi. Metaspace is capped at 1 GB, and the
JVM exits on an OutOfMemoryError so that Kubernetes restarts the container. The Iceberg REST and
Lance REST services run inside the server process and share this heap. To give the server more
memory, raise the limit and keep the request equal to it:
gravitino:
resources:
requests:
cpu: 1000m
memory: 6Gi
limits:
cpu: 2000m
memory: 6Gi
The chart refuses to render a GRAVITINO_MEM whose -Xmx plus MaxMetaspaceSize plus about 1 GiB
of native memory does not fit in the limit, and one that sets a heap without MaxMetaspaceSize.
gravitino.env replaces the chart's environment list rather than adding to it, so keep the
GRAVITINO_MEM entry when you add a variable.
Metrics
| Configuration item | Description | Default value |
|---|---|---|
gravitino.metrics.timeSlidingWindowSecs | Width in seconds of the metrics time sliding window. | 60 |
Chart value: gravitino.metrics.timeSlidingWindowSecs.
Metadata storage
Storage backend
Gravitino stores metadata over JDBC. The chart deploys PostgreSQL for the metadata database and writes
the connection properties for it, so an install needs no metadata database configuration. To use MySQL or
a database you operate, see Metadata database.
The PostgreSQL driver ships in the image. MySQL Connector/J does not, and the chart supplies it
through gravitino.jdbcDrivers.
| Configuration item | Description | Default value |
|---|---|---|
gravitino.entity.store | Entity storage implementation. relational is the only supported value. | relational |
gravitino.entity.store.relational | Relational storage implementation. JDBCBackend is the only supported value. | JDBCBackend |
gravitino.entity.store.relational.jdbcUrl | Database URL the backend connects to. | The in-chart PostgreSQL |
gravitino.entity.store.relational.jdbcDriver | Driver class name. | org.postgresql.Driver |
gravitino.entity.store.relational.jdbcUser | Database username. | gravitino |
gravitino.entity.store.relational.jdbcPassword | Database password. The chart takes it from a Secret, never from the ConfigMap. | Generated at install |
gravitino.entity.store.relational.storagePath | Where an embedded H2 database keeps its files. Unused with PostgreSQL or MySQL. | /opt/gravitino/data/jdbc |
gravitino.entity.store.relational.maxConnections | Maximum size of the JDBC connection pool. | 100 |
gravitino.entity.store.relational.maxWaitMillis | Maximum wait in milliseconds for a connection from the pool. | 1000 |
gravitino.entity.store.maxTransactionSkewTimeMs | Maximum transaction skew in milliseconds. | 2000 |
gravitino.entity.store.deleteAfterTimeMs | How long in milliseconds deleted and superseded rows are kept. Accepts 10 minutes to 30 days. | 604800000 (7 days) |
gravitino.entity.store.versionRetentionCount | Number of entity versions kept, including the current one. Accepts 1 to 10. | 1 |
Chart values: gravitino.entity.store, gravitino.entity.relationalBackend,
gravitino.entity.storagePath, gravitino.entity.maxConnections,
gravitino.entity.maxTransactionSkewTimeMs, gravitino.entity.deleteAfterTimeMs, and
gravitino.entity.versionRetentionCount. The URL, driver, user, and password come from the in-chart
database, or from gravitino.entity.jdbcUrl, jdbcDriver, jdbcUser, and jdbcPassword when
both gravitino.postgresql.enabled and gravitino.mysql.enabled are false.
Partition statistics
Partition statistics are kept in their own storage rather than
in the catalog. By default that is the partition_statistic_meta table in the metadata database,
which the schema scripts in Metadata database
already create. A custom backend is described in
Custom partition storage. These properties have no
dedicated chart values.
| Configuration item | Description | Default value |
|---|---|---|
gravitino.stats.partition.storageFactoryClass | Storage factory class. org.apache.gravitino.stats.storage.MemoryPartitionStatsStorageFactory is for testing only. | org.apache.gravitino.stats.storage.JdbcPartitionStatisticStorageFactory |
gravitino.stats.partition.storageOption.{key} | An option passed to the storage factory as {key}. | (none) |
The JDBC factory takes the options below. Each one not set falls back to the matching
gravitino.entity.store.relational.* property, so the default needs no configuration. Set them to
keep partition statistics in a different MySQL or PostgreSQL database, which must have the same
schema loaded.
| Option | Description | Default value |
|---|---|---|
jdbcUrl | Database URL. | The metadata database's jdbcUrl |
jdbcUser | Database username. | The metadata database's jdbcUser |
jdbcPassword | Database password. | The metadata database's jdbcPassword |
jdbcDriver | Driver class name. | The metadata database's jdbcDriver |
poolMaxSize | Maximum size of the connection pool. | The metadata database's maxConnections |
poolMinIdle | Minimum idle connections in the pool. | 2 |
connectionTimeoutMs | Maximum wait in milliseconds for a connection. | The metadata database's maxWaitMillis |
testOnBorrow | Whether to test a connection before use. | true |
The Lance factory, org.apache.gravitino.stats.storage.LancePartitionStatisticStorageFactory, keeps
partition statistics in a Lance dataset. It takes the options below, and passes
every option on to Lance as a storage option, so the
Lance storage options for remote object
stores go here too.
| Option | Description | Default value |
|---|---|---|
location | Where the Lance dataset is kept. | ${GRAVITINO_HOME}/data/lance |
maxRowsPerFile | Maximum rows per file. | 1000000 |
maxBytesPerFile | Maximum bytes per file. | 104857600 |
maxRowsPerGroup | Maximum rows per row group. | 1000000 |
readBatchSize | Rows read per batch. | 10000 |
datasetCacheSize | Datasets cached. 0 disables the cache. | 0 |
metadataFileCacheSizeBytes | Lance metadata file cache size in bytes. | 102400 |
indexCacheSizeBytes | Lance index cache size in bytes. | 102400 |
maxStatisticsPerUpdate | Maximum statistics in one update. | 100 |
With many tables that each have few partitions, set smaller metadataFileCacheSizeBytes and
indexCacheSizeBytes.
Entity cache
The server caches entities in memory to avoid reading the backend on every request. Caching is on by default, and the properties below tune what it holds and how it evicts.
| Configuration item | Description | Default value |
|---|---|---|
gravitino.cache.enabled | Whether to cache entities at all. | true |
gravitino.cache.implementation | Cache implementation. Use the short name, not a fully qualified class name. | caffeine |
gravitino.cache.maxEntries | Maximum number of cached entries. Ignored when enableWeigher is true. | 10000 |
gravitino.cache.expireTimeInMs | Time to live in milliseconds, measured from entry creation. | 3600000 (1 hour) |
gravitino.cache.enableWeigher | Whether to evict by weight rather than by entry count. | true |
gravitino.cache.enableStats | Whether to log hit count, miss count, and load failures every five minutes at INFO. | false |
gravitino.cache.lockSegments | Number of lock segments used to reduce contention. | 16 |
Chart values: gravitino.cache.{key} for every row except lockSegments.
Two eviction limits apply at once. Time to live always applies: an entry older than
expireTimeInMs expires and is cleaned up asynchronously. Alongside it, the cache bounds its size
either by count or by weight. With enableWeigher disabled, Caffeine's W-TinyLFU policy evicts the
least-used entries once maxEntries is reached. With enableWeigher enabled, each entity type
carries a weight, larger for entities higher in the hierarchy, and eviction targets a total weight
budget instead; maxEntries is ignored, and a single entry heavier than the whole budget is never
cached.
Change log propagation
Caches are local to each server, so a metalake modified on one server would otherwise stay stale on its neighbors. Every server writes its changes to an entity change log table and polls that table to invalidate what other servers have touched. A separate cleaner trims old rows. These properties have no dedicated chart values.
| Configuration item | Description | Default value |
|---|---|---|
gravitino.entityChangeLog.pollIntervalSecs | Interval in seconds between polls. Must be positive. | 3 |
gravitino.entityChangeLog.retentionSecs | How long in seconds change log rows are kept, measured by database time. 0 disables cleanup; otherwise use at least ten times pollIntervalSecs. | 2592000 (30 days) |
gravitino.entityChangeLog.cleanupIntervalSecs | Interval in seconds between cleaner runs. Must be positive. | 86400 (1 day) |
Each polled batch is handed to the server's caches once and is never retried. A cache that fails to apply a batch logs an error and clears itself, so nothing stale is left behind, and the server keeps serving.
Tree lock
Gravitino serializes conflicting metadata operations with an in-memory tree lock. It is the only lock implementation available, and it is per-server.
| Configuration item | Description | Default value |
|---|---|---|
gravitino.lock.maxNodes | Maximum tree lock nodes held in memory. | 100000 |
gravitino.lock.minNodes | Minimum tree lock nodes held in memory. | 1000 |
gravitino.lock.cleanIntervalInSecs | Interval in seconds for reclaiming stale lock nodes. | 60 |
Chart values: gravitino.lock.maxNodes and gravitino.lock.minNodes.
Catalog isolation
These properties govern how the server loads and isolates catalogs. The properties that configure an individual catalog are covered under Catalog properties.
credential.backfillToProperties below is the escape hatch for connectors that cannot consume
vended credentials; the mechanism it opts out of is described in
Credential vending.
| Configuration item | Description | Default value |
|---|---|---|
gravitino.catalog.cache.evictionIntervalMs | Interval in milliseconds before an idle catalog is evicted from the catalog cache. | 3600000 |
gravitino.catalog.classloader.isolated | Whether to load each catalog's libraries and configuration in an isolated classloader rather than the application classloader. | true |
gravitino.catalog.classloader.sharing.enabled | Whether catalogs whose isolation-relevant properties match may share one classloader. Sharing reduces Metaspace usage; disabling it gives every catalog its own. | true |
gravitino.catalog.credential.backfillToProperties | Whether to return hidden catalog credentials such as jdbc-user and jdbc-password in the catalog properties response, for connectors that cannot consume vended credentials. Anyone who can read catalog properties can then read those credentials. Turn it off once your connectors are upgraded. | false |
Chart values: gravitino.catalog.evictionIntervalMs and gravitino.catalog.classloaderSharing.
Server security
Authentication
Authentication decides who a caller is. The chart sets gravitino.authenticators to basic, so
every caller signs in, and refuses to render with simple, which performs no authentication and is
not supported in Datastrato Enterprise. Set it to "oauth,basic" to sign in through your identity
provider.
| Configuration item | Description | Default value |
|---|---|---|
gravitino.authenticators | Comma-separated authenticators to enable: basic, oauth, or both. | basic |
Chart value: gravitino.authenticators.
Naming an authenticator is one line; configuring it is not. Each value reads its own family of
gravitino.authenticator.* properties, covered in
Authentication. To hold users, password hashes, and group
membership in Gravitino's own relational store rather than an external provider, see
Directory.
gravitino.authenticator, in the singular, is a deprecated spelling that still works.
Authorization
Authorization decides what an authenticated caller may do. The chart turns it on and names admin,
the local admin account, as the service admin. Leaving gravitino.authorization.serviceAdmins
empty while authorization is on fails the Helm render.
| Configuration item | Description | Default value |
|---|---|---|
gravitino.authorization.enable | Whether to enforce privileges on metadata operations. | true |
gravitino.authorization.serviceAdmins | Comma-separated users who administer the service. Metalake creation is restricted to them. | admin |
gravitino.authorization.impl | Authorizer implementation. | org.apache.gravitino.server.authorization.jcasbin.JcasbinAuthorizer |
gravitino.authorization.threadPoolSize | Threads serving authorization checks. | 100 |
Chart values: gravitino.authorization.enable and gravitino.authorization.serviceAdmins.
The privilege model these properties switch on, roles, grants, ownership, and the
gravitino.authorization.jcasbin.* tuning of the default authorizer, is described in
Privileges. To push enforcement down into the underlying systems
through Apache Ranger or a native permission model, see
Authorization pushdown.
Remote files
The server fetches files by URI in two situations: staging a job's files, and loading catalog files such as a Kerberos keytab. Both accept remote URIs, so both are a path by which a caller who can create a catalog or submit a job could make the server issue requests of its own.
| Configuration item | Description | Default value |
|---|---|---|
gravitino.fetchFile.blockUnsafeRemoteUri | Whether to refuse remote URIs that resolve to unsafe addresses. Disable only for trusted URIs that need them. | true |
Chart value: gravitino.security.blockUnsafeRemoteUri.
Sensitive property names
| Configuration item | Description | Default value |
|---|---|---|
gravitino.secret.sensitiveKeyKeywords | Comma-separated keywords that mark a property name as a credential. A catalog property its catalog does not declare is treated as a secret when its name contains one, ignoring case. A list you set replaces the default one. | secret, password, token, credential, access, account, and common misspellings |
Chart value: gravitino.security.sensitiveKeyKeywords.
Audit log
The audit log framework has two halves. A formatter turns an Event into an AuditLog, and a
writer puts that AuditLog somewhere. Both are interfaces, so a deployment with its own log
pipeline can replace either.
| Configuration item | Description | Default value |
|---|---|---|
gravitino.audit.enabled | Whether to write an audit log. | true |
gravitino.audit.formatter.className | Formatter class name. | org.apache.gravitino.audit.v2.SimpleFormatterV2 |
gravitino.audit.writer.className | Writer class name. | org.apache.gravitino.audit.FileAuditWriter |
Chart values: gravitino.audit.enabled, gravitino.audit.formatter.className, and
gravitino.audit.writer.className.
SimpleFormatterV2 is the default formatter. JsonAuditFormatter is available where structured
output is wanted: it emits one JSON object per line, serializes customInfo, and writes timestamps
as ISO 8601 with millisecond precision and a zone offset. Both formatters replace the value of a
sensitive customInfo key with ***. A key is sensitive when it is authorization, cookie,
x-amz-security-token, s3.access-key-id, or jdbc-password, or when it contains password,
secret, token, credential, apikey, accesskey, privatekey, auth, or signature. Both
checks ignore case, and the second ignores every character other than letters and digits, so
api-key and x_api_key match apikey. The fixed keys the server writes itself, http.method,
http.uri, http.status, auth.method, and auth.expression, are never masked.
FileAuditWriter is the default writer, and it manages no files itself. Each server pod writes
gravitino_audit.log on the logs volume under /opt/gravitino/logs/{pod_name}/, rotates it daily
and at 256 MB, and deletes rotated files once they are older than
gravitino.log4j2Properties.logMaxAge or together exceed
gravitino.log4j2Properties.auditLogMaxTotalSize, whichever comes first. The defaults are 30d
and 10GB, and logMaxAge applies to the server, lineage, and search logs as well:
gravitino:
log4j2Properties:
logMaxAge: "90d"
auditLogMaxTotalSize: "20GB"
Paths with gravitino.logPersistence.subPath, record formats, and forwarding are in
Audit log.
Earlier releases configured the writer directly through gravitino.audit.writer.file.*. Those
properties now do nothing, and FileAuditWriter logs a warning at startup if it finds any of them.
Server extensions
Event listeners
Event listeners receive the events Gravitino emits around metadata operations.
gravitino.eventListener.names has a dedicated chart value, which defaults to
search,dashboard-metrics, and every name in it needs a gravitino.eventListener.{name}.class
entry, or the server exits at startup. Registering a listener, dispatch modes, and queue behavior
are covered in Events.
Auxiliary services
An auxiliary service runs inside the Gravitino server process on its own port. The chart starts
the Gravitino IRC server and the Lance REST server, and adds scim when SCIM is on.
| Configuration item | Description | Default value |
|---|---|---|
gravitino.auxService.names | Comma-separated auxiliary services to start. iceberg-rest is the Gravitino IRC server, lance-rest the Lance REST server. | iceberg-rest,lance-rest |
Chart value: gravitino.auxService.names.
The rest of the IRC configuration, and the gravitino.lance-rest.* properties of the Lance REST
server, are documented with those services. See
Iceberg REST service.
Jobs
| Configuration item | Description | Default value |
|---|---|---|
gravitino.job.executor | Executor that runs jobs. Implement your own and name it here to replace the built-in one. | local |
gravitino.job.stagingDir | Directory holding staging files for running jobs. With gravitino.logPersistence.subPath set, the chart uses /opt/gravitino/logs/{sub_path}/jobs/staging. | /opt/gravitino/logs/jobs/staging |
gravitino.job.stagingDirKeepTimeInMs | How long in milliseconds a finished job's staging files are kept. Use at least 10 minutes outside testing. | 604800000 (7 days) |
gravitino.job.statusPullIntervalInMs | Interval in milliseconds between job status polls. Use at least 1 minute outside testing. | 60000 (1 minute) |
The built-in local executor reads properties under gravitino.jobExecutor.local.. A custom
executor is named by gravitino.jobExecutor.{name}.class and receives the properties under
gravitino.jobExecutor.{name}.; see Custom job executor.
| Configuration item | Description | Default value |
|---|---|---|
gravitino.jobExecutor.local.waitingQueueSize | Jobs that can wait for a free slot. A run submitted when the queue is full is refused. | 100 |
gravitino.jobExecutor.local.maxRunningJobs | Jobs run at once. | 1 |
gravitino.jobExecutor.local.jobStatusKeepTimeInMs | How long in milliseconds the executor keeps a finished job's status in memory. | 3600000 (1 hour) |
gravitino.jobExecutor.local.sparkHome | Spark installation used to submit Spark jobs. When unset, SPARK_HOME is used, and with neither Spark jobs fail. | (none) |
Chart values: gravitino.job.stagingDir, gravitino.job.stagingDirKeepTimeInMs,
gravitino.job.statusPullIntervalInMs, gravitino.job.maxRunningJobs for
gravitino.jobExecutor.local.maxRunningJobs, and gravitino.job.sparkHome for
gravitino.jobExecutor.local.sparkHome.
Catalog properties
Catalog properties configure one catalog rather than the server. They are set in the properties
of a catalog when you create it, through the UI or the API, and apply to that catalog alone. They
do not affect schema or table properties.
A catalog property is one of three kinds. Gravitino defines some itself, as the settings a catalog
needs to work. Anything prefixed gravitino.bypass. passes straight through to the underlying
system untouched. Anything else Gravitino simply stores for you to use as you like.
Passing credentials, tokens, or access keys through gravitino.bypass. exposes them: bypassed
properties are not managed by Gravitino and can come back in plaintext from the REST API. Where an
underlying system leaves no alternative, restrict access to the catalog APIs accordingly.
These properties apply to every catalog:
| Configuration item | Description | Default value |
|---|---|---|
package | Path to the catalog package, from which Gravitino loads the catalog's libraries and configuration. It holds a conf directory and a libs directory. | (none) |
cloud.name | Cloud the catalog runs on. One of aws, azure, gcp, on_premise, or other. | (none) |
cloud.region-code | Region code within that cloud. | (none) |
Everything else is per provider:
| Catalog provider | Catalog properties |
|---|---|
hive | Hive catalog properties |
glue | AWS Glue catalog properties |
lakehouse-iceberg | Lakehouse Iceberg catalog properties |
lakehouse-paimon | Lakehouse Paimon catalog properties |
lakehouse-hudi | Lakehouse Hudi catalog properties |
lakehouse-generic | Lakehouse Generic catalog properties |
jdbc-mysql | MySQL catalog properties |
jdbc-postgresql | PostgreSQL catalog properties |
jdbc-oracle | Oracle catalog properties |
jdbc-sqlserver | SQL Server catalog properties |
jdbc-bigquery | BigQuery catalog properties |
jdbc-doris | Doris catalog properties |
jdbc-starrocks | StarRocks catalog properties |
jdbc-clickhouse | ClickHouse catalog properties |
jdbc-hologres | Hologres catalog properties |
jdbc-oceanbase | OceanBase catalog properties |
kafka | Kafka catalog properties |
fileset | Fileset catalog properties |
model | Model catalog properties |
Apache Hadoop access
Gravitino reaches Hadoop as a single operating system user, so that user needs the HDFS and YARN
permissions for everything the server will touch. Without them, operations fail with
Permission denied. Set HADOOP_USER_NAME to a user that has them, in gravitino.env, keeping
the GRAVITINO_MEM entry in the same list:
gravitino:
env:
- name: GRAVITINO_MEM
value: "-XX:MaxRAMPercentage=50 -XX:MaxMetaspaceSize=1g -XX:+ExitOnOutOfMemoryError"
- name: HADOOP_USER_NAME
value: "{hadoop_user}"