Spark authentication
Overview
Spark connector supports simple, basic, oauth2, and kerberos authentication when accessing Gravitino server.
Datastrato Enterprise does not support the simple authenticator, so use Basic, OAuth, or Kerberos.
| Property | Type | Default Value | Description | Required |
|---|---|---|---|---|
| spark.sql.gravitino.authType | string | simple | The authentication mechanism for communicating with the Gravitino server. Supported values: simple, basic, oauth2, kerberos. | No |
Basic mode
In Basic mode, the Spark connector authenticates to the Gravitino server using HTTP Basic credentials against the local user store. The Gravitino server must have Basic authentication enabled. See How to authenticate for server-side setup.
| Property | Type | Default Value | Description | Required |
|---|---|---|---|---|
| spark.sql.gravitino.authType | string | simple | Set to basic to enable Basic authentication. | Yes, for Basic mode |
| spark.sql.gravitino.basic.username | string | (none) | Username in the local user store. | Yes, for Basic mode |
| spark.sql.gravitino.basic.password | string | (none) | Password for that user. | Yes, for Basic mode |
Basic configuration example
spark.plugins=org.apache.gravitino.spark.connector.plugin.GravitinoSparkPlugin
spark.sql.gravitino.uri=http://localhost:8090
spark.sql.gravitino.metalake=my_metalake
spark.sql.gravitino.authType=basic
spark.sql.gravitino.basic.username=admin
spark.sql.gravitino.basic.password=YourSecureGravitinoPassword
OAuth mode
In the OAuth mode, you could use the following configuration to fetch an OAuth token to access Gravitino server.
| Property | Type | Default Value | Description | Required |
|---|---|---|---|---|
| spark.sql.gravitino.oauth2.serverUri | string | None | The OAuth server uri address. | Yes, for OAuth mode |
| spark.sql.gravitino.oauth2.tokenPath | string | None | The path of token interface in OAuth server. | Yes, for OAuth mode |
| spark.sql.gravitino.oauth2.credential | string | None | The credential to request the OAuth token. | Yes, for OAuth mode |
| spark.sql.gravitino.oauth2.scope | string | None | The scope to request the OAuth token. | Yes, for OAuth mode |
Kerberos mode
In Kerberos mode, you could use the Spark Kerberos configuration to fetch a Kerberos ticket to access Gravitino server, use spark.kerberos.principal, spark.kerberos.keytab to specify Kerberos principal and keytab.
The Gravitino server principal has the form HTTP/$host@$realm. Keep $host consistent with the host in the Gravitino server URI.
Please make sure krb5.conf is accessible by Spark, like by specifying the configuration spark.driver.extraJavaOptions="-Djava.security.krb5.conf=/xx/krb5.conf".