Manage User-Defined Functions

Introduction

This page covers the Gravitino API for functions. For what a function is, the function types, determinism, and how definitions and implementations relate, see Functions. For creating the catalog and schema a function lives in, see Manage Catalogs and Schemas.

Function Operations

Register a SQL Function

A function needs a name, a type, a determinism flag, and at least one definition. A definition carries its parameters, its return type, and one or more implementations.

REST

curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
  -H "Content-Type: application/json" -d '{
  "name": "add_one",
  "functionType": "SCALAR",
  "deterministic": true,
  "comment": "Adds one to the input",
  "definitions": [
    {
      "parameters": [{"name": "x", "dataType": "integer"}],
      "returnType": "integer",
      "impls": [
        {"language": "SQL", "runtime": "TRINO", "sql": "x + 1"}
      ]
    }
  ]
}' http://localhost:8090/api/metalakes/example/catalogs/sales/schemas/public/functions

Java

Catalog catalog = client.loadCatalog("sales");
FunctionCatalog functions = catalog.asFunctionCatalog();

FunctionImpl sqlImpl = FunctionImpls.ofSql(FunctionImpl.RuntimeType.TRINO, "x + 1");

FunctionDefinition definition = FunctionDefinitions.of(
    new FunctionParam[] {FunctionParams.of("x", Types.IntegerType.get())},
    Types.IntegerType.get(),
    new FunctionImpl[] {sqlImpl});

Function function = functions.registerFunction(
    NameIdentifier.of("public", "add_one"),
    "Adds one to the input",
    FunctionType.SCALAR,
    true,
    new FunctionDefinition[] {definition});

Python

catalog = client.load_catalog("sales")
functions = catalog.as_function_catalog()

sql_impl = (
    SQLImpl.builder()
    .with_runtime_type(SQLImpl.RuntimeType.TRINO)
    .with_sql("x + 1")
    .build()
)

definition = FunctionDefinitions.of(
    [FunctionParams.of("x", Types.IntegerType.get())],
    Types.IntegerType.get(),
    [sql_impl])

function = functions.register_function(
    ident=NameIdentifier.of("public", "add_one"),
    comment="Adds one to the input",
    function_type=FunctionType.SCALAR,
    deterministic=True,
    definitions=[definition])

Register a Python Function

A Python implementation names a handler entrypoint, and can carry inline code and the packages the runtime needs.

REST

curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
  -H "Content-Type: application/json" -d '{
  "name": "normalize_phone",
  "functionType": "SCALAR",
  "deterministic": true,
  "comment": "Strips formatting from a phone number",
  "definitions": [
    {
      "parameters": [{"name": "raw", "dataType": "varchar(64)"}],
      "returnType": "varchar(64)",
      "impls": [
        {
          "language": "PYTHON",
          "runtime": "SPARK",
          "handler": "normalize.main",
          "codeBlock": "def main(raw):\n    return \"\".join(c for c in raw if c.isdigit())"
        }
      ]
    }
  ]
}' http://localhost:8090/api/metalakes/example/catalogs/sales/schemas/public/functions

Register a Java Function

A Java implementation names a class, and usually the jar that holds it. A table-valued function declares returnColumns rather than a single returnType.

REST

curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
  -H "Content-Type: application/json" -d '{
  "name": "generate_series",
  "functionType": "TABLE",
  "deterministic": true,
  "comment": "Generates a range of integers",
  "definitions": [
    {
      "parameters": [
        {"name": "start_val", "dataType": "integer"},
        {"name": "end_val", "dataType": "integer"}
      ],
      "returnColumns": [
        {"name": "value", "dataType": "integer", "comment": "The generated value"}
      ],
      "impls": [
        {
          "language": "JAVA",
          "runtime": "SPARK",
          "className": "com.example.GenerateSeriesFunction",
          "resources": {"jars": ["hdfs:///path/to/udtf.jar"]}
        }
      ]
    }
  ]
}' http://localhost:8090/api/metalakes/example/catalogs/sales/schemas/public/functions

Register Overloads

A function with several definitions accepts several parameter lists under one name. Each definition carries its own return type and implementations.

REST

curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
  -H "Content-Type: application/json" -d '{
  "name": "add",
  "functionType": "SCALAR",
  "deterministic": true,
  "comment": "Adds two or three integers",
  "definitions": [
    {
      "parameters": [
        {"name": "x", "dataType": "integer"},
        {"name": "y", "dataType": "integer"}
      ],
      "returnType": "integer",
      "impls": [{"language": "SQL", "runtime": "TRINO", "sql": "x + y"}]
    },
    {
      "parameters": [
        {"name": "x", "dataType": "integer"},
        {"name": "y", "dataType": "integer"},
        {"name": "z", "dataType": "integer"}
      ],
      "returnType": "integer",
      "impls": [{"language": "SQL", "runtime": "TRINO", "sql": "x + y + z"}]
    }
  ]
}' http://localhost:8090/api/metalakes/example/catalogs/sales/schemas/public/functions

Get a Function

REST

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
  http://localhost:8090/api/metalakes/example/catalogs/sales/schemas/public/functions/add_one

Java

Function function = functions.getFunction(NameIdentifier.of("public", "add_one"));

Python

function = functions.get_function(NameIdentifier.of("public", "add_one"))

List Functions

REST

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
  http://localhost:8090/api/metalakes/example/catalogs/sales/schemas/public/functions

Java

NameIdentifier[] identifiers = functions.listFunctions(Namespace.of("public"));

Python

identifiers = functions.list_functions(Namespace.of("public"))

Alter a Function

Changes are applied as a list in one request.

Change	JSON	Java
Rename	{"@type":"rename","newName":"add_one_v2"}	FunctionChange.rename("add_one_v2")
Update the comment	{"@type":"updateComment","newComment":"new_comment"}	FunctionChange.updateComment("new_comment")
Set a property	{"@type":"setProperty","property":"key1","value":"value1"}	FunctionChange.setProperty("key1", "value1")
Remove a property	{"@type":"removeProperty","property":"key1"}	FunctionChange.removeProperty("key1")

REST

curl -X PUT -H "Accept: application/vnd.gravitino.v1+json" \
  -H "Content-Type: application/json" -d '{
  "updates": [
    {"@type": "updateComment", "newComment": "Adds one, reviewed"}
  ]
}' http://localhost:8090/api/metalakes/example/catalogs/sales/schemas/public/functions/add_one

Java

Function function = functions.alterFunction(
    NameIdentifier.of("public", "add_one"),
    FunctionChange.updateComment("Adds one, reviewed"));

Drop a Function

REST

curl -X DELETE -H "Accept: application/vnd.gravitino.v1+json" \
  http://localhost:8090/api/metalakes/example/catalogs/sales/schemas/public/functions/add_one

Java

boolean dropped = functions.dropFunction(NameIdentifier.of("public", "add_one"));

Python

dropped = functions.drop_function(NameIdentifier.of("public", "add_one"))