Skip to main content
Version: 0.4.0

Gravitino configuration

Introduction

Gravitino supports several configurations:

  1. Gravitino server configuration: Used to start up the Gravitino server.
  2. Gravitino catalog properties configuration: Used to make default values for different catalogs.
  3. Some other configurations: Includes HDFS and other configurations.

Gravitino server configurations

You can customize the Gravitino server by editing the configuration file gravitino.conf in the conf directory. The default values are sufficient for most use cases. We strongly recommend that you read the following sections to understand the configuration file so you can change the default values to suit your specific situation and usage scenario.

The gravitino.conf file lists the configuration items in the following table. It groups those items into the following categories:

Gravitino HTTP Server configuration

Configuration itemDescriptionDefault valueRequiredSince version
gravitino.server.webserver.hostThe host of the Gravitino server.0.0.0.0No0.1.0
gravitino.server.webserver.httpPortThe port on which the Gravitino server listens for incoming connections.8090No0.1.0
gravitino.server.webserver.minThreadsThe minimum number of threads in the thread pool used by the Jetty webserver. minThreads is 8 if the value is less than 8.Math.max(Math.min(Runtime.getRuntime().availableProcessors() * 2, 100), 8)No0.2.0
gravitino.server.webserver.maxThreadsThe maximum number of threads in the thread pool used by the Jetty webserver. maxThreads is 8 if the value is less than 8, and maxThreads must be great or equal to minThreads.Math.max(Runtime.getRuntime().availableProcessors() * 4, 400)No0.1.0
gravitino.server.webserver.threadPoolWorkQueueSizeThe size of the queue in the thread pool used by the Jetty webserver.100No0.1.0
gravitino.server.webserver.stopTimeoutTime in milliseconds to gracefully shut down the Jetty webserver, for more, please see org.eclipse.jetty.server.Server#setStopTimeout.30000No0.2.0
gravitino.server.webserver.idleTimeoutThe timeout in milliseconds of idle connections.30000No0.2.0
gravitino.server.webserver.requestHeaderSizeMaximum size of HTTP requests.131072No0.1.0
gravitino.server.webserver.responseHeaderSizeMaximum size of HTTP responses.131072No0.1.0
gravitino.server.shutdown.timeoutTime in milliseconds to gracefully shut down of the Gravitino webserver.3000No0.2.0
gravitino.server.webserver.customFiltersComma-separated list of filter class names to apply to the API.(none)No0.4.0

The filter in the customFilters should be a standard javax servlet filter. You can also specify filter parameters by setting configuration entries of the form gravitino.server.webserver.<class name of filter>.param.<param name>=<value>.

Storage configuration

Configuration itemDescriptionDefault valueRequiredSince version
gravitino.entity.storeWhich storage implementation to use. Key-value pair storage is currently supported, and the default value is kv.kvNo0.1.0
gravitino.entity.store.kvDetailed implementation of KV storage. RocksDB storage is currently supported, and the implementation is RocksDBKvBackend.RocksDBKvBackendNo0.1.0
gravitino.entity.store.kv.rocksdbPathDirectory path of RocksDBKvBackend.${GRAVITINO_HOME}/data/rocksdbNo0.1.0
graivitino.entity.serdeThe serialization/deserialization class used to support entity storage. `proto' is currently supported.protoNo0.1.0
gravitino.entity.store.maxTransactionSkewTimeMsThe maximum skew time of transactions in milliseconds.2000No0.3.0
gravitino.entity.store.kv.deleteAfterTimeMsThe maximum time in milliseconds that deleted and old-version data is kept. Set to at least 10 minutes and no longer than 30 days.604800000(7 days)No0.3.0
caution

We strongly recommend that you change the default value of gravitino.entity.store.kv.rocksdbPath, as it's under the deployment directory and future version upgrades may remove it.

Catalog configuration

Configuration itemDescriptionDefault valueRequiredSince version
gravitino.catalog.cache.evictionIntervalMsThe interval in milliseconds to evict the catalog cache; default 3600000ms(1h).3600000No0.1.0
gravitino.catalog.classloader.isolatedWhether to use an isolated classloader for catalog. If true, an isolated classloader loads all catalog-related libraries and configurations, not the AppClassLoader. The default value is true.trueNo0.1.0

Auxiliary service configuration

Configuration itemDescriptionDefault valueSince Version
gravitino.auxService.names The auxiliary service name of the Gravitino Iceberg REST server. Use iceberg-rest for the Gravitino Iceberg REST server.(none)0.2.0

Refer to Iceberg REST catalog service for configuration details.

Security configuration

Refer to security for HTTPS and authentication configurations.

Gravitino catalog properties configuration

There are three types of catalog properties:

  1. Gravitino-defined properties: These properties simplify the catalog creation process.
  2. Properties with the gravitino.bypass. prefix: These properties aren't managed by Gravitino; instead, they bypass the underlying system for advanced usage.
  3. Other properties: Stored in Gravitino storage, these properties don't bypass the underlying system.

Catalog properties are either defined in catalog configuration files as default values or specified explicitly when creating a catalog.

info

Explicit specifications take precedence over formal configurations.

caution

These rules only apply to the catalog properties and don't affect the schema or table properties.

catalog providercatalog propertiescatalog properties configuration file path
hiveHive catalog propertiescatalogs/hive/conf/hive.conf
lakehouse-icebergLakehouse Iceberg catalog propertiescatalogs/lakehouse-iceberg/conf/lakehouse-iceberg.conf
jdbc-mysqlMySQL catalog propertiescatalogs/jdbc-mysql/conf/jdbc-mysql.conf
jdbc-postgresqlPostgreSQL catalog propertiescatalogs/jdbc-postgresql/conf/jdbc-postgresql.conf
info

The Gravitino server automatically adds the catalog properties configuration directory to classpath.

Some other configurations

You could put HDFS configuration file to the catalog properties configuration dir, like catalogs/lakehouse-iceberg/conf/.

How to set up runtime environment variables

The Gravitino server lets you set up runtime environment variables by editing the gravitino-env.sh file, located in the conf directory.

How to access Apache Hadoop

Currently, due to the absence of a comprehensive user permission system, Gravitino can only use a single username for Apache Hadoop access. Ensure that the user starting the Gravitino server has Hadoop (HDFS, YARN, etc.) access permissions; otherwise, you may encounter a Permission denied error. There are two ways to resolve this error:

  • Grant Gravitino startup user permissions in Hadoop
  • Specify the authorized Hadoop username in the environment variables HADOOP_USER_NAME before starting the Gravitino server.