Manage Messaging Metadata

Introduction

This page covers the Gravitino API for topics. For what a topic is, the default schema a messaging catalog presents, topic properties, and what Gravitino does not cover, see Topics. For creating the catalog a topic lives in, see Manage Catalogs and Schemas and Apache Kafka catalog.

A messaging catalog presents a single schema named default, so every path below uses it.

The Python client does not cover topics. as_topic_catalog() exists but raises UnsupportedOperationException, so the examples below are REST and Java only.

Topic Operations

Create a Topic

A topic needs a name. partition-count and replication-factor are optional and fall back to the broker's own defaults.

REST

curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
  -H "Content-Type: application/json" -d '{
  "name": "orders",
  "comment": "Order events",
  "properties": {
    "partition-count": "3",
    "replication-factor": "1"
  }
}' http://localhost:8090/api/metalakes/example/catalogs/events/schemas/default/topics

Java

Catalog catalog = client.loadCatalog("events");
TopicCatalog topics = catalog.asTopicCatalog();

Topic topic = topics.createTopic(
    NameIdentifier.of("default", "orders"),
    "Order events",
    null,
    ImmutableMap.of("partition-count", "3", "replication-factor", "1"));

The third argument to createTopic is the message schema, which is not supported and is always null.

Load a Topic

REST

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
  http://localhost:8090/api/metalakes/example/catalogs/events/schemas/default/topics/orders

Java

Topic topic = topics.loadTopic(NameIdentifier.of("default", "orders"));

Alter a Topic

Changes are applied as a list in one request. partition-count can be increased through a property change; replication-factor is immutable once the topic exists.

Change	JSON	Java
Update the comment	{"@type":"updateComment","newComment":"new_comment"}	TopicChange.updateComment("new_comment")
Set a property	{"@type":"setProperty","property":"key1","value":"value1"}	TopicChange.setProperty("key1", "value1")
Remove a property	{"@type":"removeProperty","property":"key1"}	TopicChange.removeProperty("key1")

REST

curl -X PUT -H "Accept: application/vnd.gravitino.v1+json" \
  -H "Content-Type: application/json" -d '{
  "updates": [
    {"@type": "setProperty", "property": "partition-count", "value": "6"}
  ]
}' http://localhost:8090/api/metalakes/example/catalogs/events/schemas/default/topics/orders

Java

Topic topic = topics.alterTopic(
    NameIdentifier.of("default", "orders"),
    TopicChange.setProperty("partition-count", "6"));

Drop a Topic

Dropping a topic through Gravitino deletes it in the cluster, along with its messages.

REST

curl -X DELETE -H "Accept: application/vnd.gravitino.v1+json" \
  http://localhost:8090/api/metalakes/example/catalogs/events/schemas/default/topics/orders

Java

boolean dropped = topics.dropTopic(NameIdentifier.of("default", "orders"));

List Topics

REST

curl -X GET -H "Accept: application/vnd.gravitino.v1+json" \
  http://localhost:8090/api/metalakes/example/catalogs/events/schemas/default/topics

Java

NameIdentifier[] identifiers = topics.listTopics(Namespace.of("default"));