Skip to main content

Model catalog

A model catalog is a registry for machine learning models. It follows the same three-level namespace as every other catalog, catalog then schema then model, and adds a fourth level beneath each model for its versions.

What it stores is the mapping from a name to a location. Instead of jobs and notebooks carrying a path to a model file, they resolve a model by name and get the version they asked for. The model files themselves stay wherever they live.

Models sit in the same hierarchy as the tables they were trained on, carry tags and policies like anything else, and are governed by the same roles and privileges.

Gravitino manages this catalog itself rather than federating an external registry, so a model catalog takes no provider.

Quick start​

The example creates the model catalog ml and the schema fraud in it, registers the model fraud_detector, and links its first version to the files at s3://ml-models/fraud/v1 under the alias prod. Gravitino records where each version lives; it does not copy or store the model files.

  1. Create the catalog. Open Connect → Catalogs, click + Catalog, enter ml as the Catalog name, and choose Model as the Catalog type. A model catalog needs no connection details. Click Create catalog.

  2. Create the schema. In Explore, open ml, click + Schema, and create fraud.

  3. Register the model. Open fraud, click + Model, enter fraud_detector as the Model name, and click Register model. Comment, Tags, and, under Advanced, Properties are optional.

  4. Link a version. Open fraud_detector, go to the Versions tab, and click Link version. Enter s3://ml-models/fraud/v1 under URIs and prod under Aliases, then click Submit. The first version linked is version 0.

The Versions tab lists each version with controls to view, edit, and delete it. Viewing a version shows its URIs, aliases, comment, and properties. The model's Overview tab shows its owner and properties, and its header shows its grants, tags, and policies, with a control for each. To tag several models at once, select their rows in the schema and click + Tags.

The model registry​

Models and versions​

A model is a named object in a schema, and holds no location of its own.

A model version is one release of that model. It carries the URI where the files live, an optional comment, its own properties, and a version number assigned in sequence starting at zero. A new release is a new version rather than an edit of an existing one.

Deleting a model deletes all of its versions and their aliases with it.

Aliases​

A version can carry aliases, and an alias resolves to a version the same way its number does. That is how a moving pointer such as production or champion is expressed: link a new version, move the alias, and everything resolving by that alias picks up the new release without any code changing.

An alias belongs to one version at a time within a model.

Several URIs on one version​

A version can carry more than one URI, each with a name, held as a map rather than a single value. That is how the same release is described in more than one place, such as a copy per region or per storage system.

default-uri-name picks which one is returned when a caller does not name one, and can be set on the version or on the model as a default for all of its versions. A URI supplied without a name is recorded as unknown.

Properties​

Neither the catalog nor its schemas have predefined properties beyond the common catalog properties. Models and versions carry free-form properties of their own, with default-uri-name reserved.

Permissions​

Registering a model takes REGISTER_MODEL on its schema, catalog, or metalake, and linking a version takes LINK_MODEL_VERSION. Reading a model's metadata and versions takes USE_MODEL, along with USE_CATALOG and USE_SCHEMA on the way down. See Privileges.

Use the API​

Models and versions can be registered, linked, listed, altered, and dropped over REST and through the Java and Python clients. The REST reference documents each call, with the client code under the Java and Python tabs.

Last updated on