August 2026
Part 2 of this series ended on a promise: engine freedom is now a config change. Four engines joined one Iceberg table, each with a handful of lines pointed at the same REST endpoint.
That shift quietly moved where the important decision lives. When every engine can reach every table through the same interface, picking an engine stops being an architecture question and becomes a workload question: use whatever fits the job, swap it when something better shows up.
The architecture question moves down a layer, to the thing all those engines share: the catalog.
This post is the decision framework we promised back in part 1. Not a feature matrix (the internet has plenty, and they go stale by the quarter), but the four questions that will still matter in five years.
Why the catalog is the sticky part
Part 1 covered what the Iceberg REST Catalog spec deliberately doesn't solve: it standardizes the interface, not the policy model. Your access rules, ownership metadata, tags, and lineage live inside whichever catalog you chose, and there is no standard for moving them between catalogs.
That's the first anchor. The second is operational gravity. Every engine config points at the catalog. Every pipeline authenticates through it. With credential vending, it's also the thing handing out storage credentials, which makes it your security chokepoint.
Moving tables out is doable (the spec even gives you a register path). Moving the governance you built around those tables is a rebuild, not a migration.
Vendors have noticed, too. A catalog burns almost no compute, while engines burn plenty, so the going strategy is to give the catalog away and monetize everything that connects to it. Read that pricing pattern for what it says: the catalog is the strategic spear-tip now, for you and for them.
So the tables are portable. The catalog defining the tables is not. Engine decisions come back around every year or two; this one might come around once.
That longevity is the whole argument for giving this one decision more deliberate attention than it usually gets: evaluate it on the axes that persist, not on this quarter's feature checklist.
Four questions that outlive the feature checklist
1. Catalog federation versus Iceberg only
The REST spec speaks Iceberg tables, full stop. Your Kafka topics, your Postgres tables, the fileset full of training data, the ML models: none of that is addressable through an Iceberg REST endpoint (part 1's second edge).
An Iceberg-only catalog is a clean choice if your enterprise is genuinely Iceberg-only. Most are not.
When they aren't, the gap gets covered one of two ways. Every source grows its own catalog, each with its own policy and access model, and you rebuild governance as many times as you have systems. Or one federated layer registers the catalogs you already run and speaks for the whole system, and policy gets defined in one place. That registration claim is checkable, and we check it below.
If your platform goes beyond Iceberg, weigh federation first: today's clean single-format choice is next year's many-catalogs problem. (Federation at enterprise scale gets its own pillar later in this series.)
2. Semantic layers and ontologies
A catalog's initial job was answering where the data is. Now its role is growing into answering what the data means. Take a table called orders: the catalog knows its schema, location, and snapshot history.
It doesn't know that amount is gross rather than net, that the EU rows carry a retention clock, or that finance and marketing mean different things by "an order." That knowledge is real, and today it lives in a BI tool's semantic layer, a wiki, or three people's heads.
That shift matters twice over, because the consumers are no longer just humans. An AI agent can't infer that revenue_adj is the number the CFO signs off on. It needs meaning attached to metadata, in machine-readable form, at the same place the data is discovered.
Where will that meaning live? A catalog that can attach semantics across the whole estate lets you define them once. A catalog that can't passes the issue along to the consumer that needs it, and your agents inherit the fragmentation. You don't need a full ontology on day one. You need a catalog with somewhere to put one.
3. Data residency
Where is your data allowed to live, and where does it actually live? For modern enterprises, the honest answer is often: on-prem for the systems that predate the cloud, two clouds by way of an acquisition, and a region requirement from a regulator.
The catalog has to be able to run where the data lives, and metadata is data too: table names, schemas, and lineage can be as sensitive as the rows themselves. If the catalog only runs in one vendor's cloud, your metadata lives there no matter where your data has to be. A catalog you can deploy yourself runs wherever the requirement points: on-prem, another cloud, a pinned region.
When a workload has to move next year, on-prem, into another cloud, or into a pinned region, does the catalog come along, or is it the reason you can't?
4. Catalog only, or platform provided
Ask who the catalog works for.
A platform's catalog answers to the platform: it exists to feed compute, its roadmap is scored on how much of the platform you use, and it will be excellent right up to the platform's edge and indifferent past it.
A catalog-only tool answers to its users, because there is nothing else to feed. Working with every engine, every format, and every cloud isn't a roadmap item; it's the entire business.
Managed versus self-hosted is the debate teams usually have, and both can be run well. Who the catalog answers to is the one you can't operate your way around.
That incentive shows up somewhere concrete: provisioning. On platform catalogs, a catalog is something you are issued. It exists before you, there's one of it (maybe one per account, maybe one per region), and getting another is a procurement conversation.
On the other side, a catalog is an API object: you create as many as you want, per team, per environment, per tenant, the way you'd create a namespace or a table.
The sharpest test here is still the exit question: if we left in three years, what would we have to rebuild? And whatever you choose, part 2's advice stands: the catalog referees every commit and vends every credential, so run it like tier-one infrastructure.
Catalogs as API objects
Two of the four questions can be proven in a terminal, so let's prove them, starting with question 4's contrast. The setup is the same Apache Gravitino stack from part 2 (Iceberg REST service in dynamic mode), and everything below is in the series repo at github.com/datastrato/blog-iceberg-in-practice, under part-3-catalog-choice/, independently runnable from a fresh clone.
One piece of housekeeping first, and it's a teaching point, not a chore. A catalog is a serving and governance boundary; the backend database under it is the storage boundary. If two catalogs share one backend database, they're two names for the same namespace set, not two catalogs. So each catalog here gets its own backend database and its own warehouse prefix. One line each:
docker compose exec -T postgres psql -U iceberg -d iceberg \
-c "CREATE DATABASE iceberg_team_b"
Now the claim. Creating a catalog is one POST to Gravitino's management API:
curl -X POST http://gravitino:8090/api/metalakes/demo_metalake/catalogs \
-H 'Content-Type: application/json' \
-d '{
"name": "team_b_dev",
"type": "RELATIONAL",
"provider": "lakehouse-iceberg",
"properties": {
"catalog-backend": "jdbc",
"uri": "jdbc:postgresql://postgres:5432/iceberg_team_b",
"warehouse": "s3://lakehouse/warehouse-team-b",
"jdbc-initialize": "true",
"credential-providers": "s3-token"
}
}'
(Abbreviated for reading; the repo scripts carry the full property set, including the S3 and JDBC credentials.)
Is it real? Ask the Iceberg REST endpoint, which every engine in part 2 talks to:
curl http://catalog:9001/iceberg/v1/config?warehouse=team_b_dev
In our test runs, that returned a full config response in the same second the POST completed. No restart, no config file, no redeploy. The server's restart count over the entire verification run: zero. A namespace created through the new catalog is immediately visible; the neighboring catalog's namespaces are not, because the storage boundary underneath is real.
Teardown is where Gravitino shows its manners. Try to delete the catalog the obvious way and you get refused:
curl -X DELETE http://gravitino:8090/api/metalakes/demo_metalake/catalogs/team_b_dev
# CatalogInUseException: "Catalog demo_metalake.team_b_dev is in use,
# please disable it first or use force option"
That refusal is a feature, not a bug. Infrastructure you can create with one call is infrastructure someone will eventually delete with one call, so an active catalog can't be dropped by accident. Disable it first, then drop:
curl -X PATCH http://gravitino:8090/api/metalakes/demo_metalake/catalogs/team_b_dev \
-H 'Content-Type: application/json' -d '{"inUse": false}'
curl -X DELETE http://gravitino:8090/api/metalakes/demo_metalake/catalogs/team_b_dev
# {"code":0,"dropped":true}
The REST endpoint now returns 404 for team_b_dev; its neighbor is untouched. And one footnote that makes the whole pattern safe to use liberally: dropping a catalog de-registers it. The backend metadata and the warehouse files survive, and a new catalog pointed at the same backend picks the tables right back up (we verified that too). Catalog objects are cheap and disposable precisely because dropping one destroys nothing.
That's what "catalogs as API objects" buys you: per-team, per-environment, per-tenant catalogs stop being a procurement conversation and become a provisioning script.
Register a catalog you already run
Creating new catalogs is half the story. The other half is question 1's claim: what about the catalog you already have?
For this one we seeded a plain Iceberg JDBC catalog with PyIceberg, four rows in a sales.orders table, no Gravitino anywhere in its config. Registering it is the same POST as before, with the backend pointed at the existing database:
curl -X POST http://gravitino:8090/api/metalakes/demo_metalake/catalogs \
-H 'Content-Type: application/json' \
-d '{
"name": "legacy",
"type": "RELATIONAL",
"provider": "lakehouse-iceberg",
"properties": {
"catalog-backend": "jdbc",
"catalog-backend-name": "legacy",
"uri": "jdbc:postgresql://postgres:5432/iceberg_legacy",
"warehouse": "s3://lakehouse/warehouse-legacy",
"credential-providers": "s3-token"
}
}'
(Abbreviated again; the repo scripts carry the full property set.)
The seeded namespace and table appear through the REST endpoint immediately, and DuckDB, which had no part in writing them, reads all four rows back on vended credentials.
Gravitino never created this metadata. It registered it and served it, unchanged: no migration, no rewrite, no restart. That's federation by registration, the smallest version of it. Federating whole estates across clouds is the later pillar.
One service, two front doors
There's a structural reason the demo above works the way it does, and it answers question 1 without giving up the spec-standard interface.
Gravitino exposes two interfaces to the same metadata. The first is the Iceberg REST API we discussed in part 2: spec-standard, speaks to any IRC client, which is how DuckDB and PyIceberg connected with no Gravitino-specific anything.
The second is the native API: the management surface where catalogs, metalakes, and non-Iceberg assets live, with engine connectors and a Python client on top.
Iceberg workloads get the standard interface the whole ecosystem converged on. The world beyond Iceberg gets a metadata plane above any single format. Same service, same referee, two doors.
Two interfaces, one service: spec-standard Iceberg REST for IRC clients, the native API for the rest of the estate.
The takeaway
Run the four questions and not everyone lands in the same place, which is the point of a framework.
If nothing is forcing a move, the metastore you already run remains a legitimate answer. If your estate is Iceberg-only, an Iceberg-native catalog is a clean fit; just check how long "Iceberg-only" survived contact with streaming topics, model registries, and the odd operational database. And if you're all-in on one vendor's platform, their catalog is the path of least resistance; price question 4 into it before calling it permanent, because the governance you build there doesn't leave with you.
Notice what happens when more than one question is live at once, though: an estate that spans formats, meaning you'd rather define once, residency you can't negotiate, teams that want their own catalogs without a ticket queue. Those answers converge on a particular shape of catalog:
- Federation-first, not Iceberg-only. It registers the catalogs you already run and speaks for the whole estate.
- A home for meaning. Semantics attach where data is discovered, defined once, for people and agents alike.
- Runs where your data must live. On-prem, any cloud, pinned regions: deployment stays your call.
- Catalog-only and neutral by design. No platform it serves first, and catalogs are objects you create, not endpoints you're issued.
The demos above showed that shape live in a terminal, and the difference it adds up to is this: choosing a catalog for what it does today versus choosing one for what it lets you keep deciding later. Whichever way you land, answer deliberately. The catalog you choose by default is still a choice, just one made without asking what it costs to leave.
The thing to do: clone the series repo, run the part-3-catalog-choice/ tour, and create, register, and destroy catalogs in a few minutes flat. Then run the four questions against whatever catalog you're on today. Write the answers down; future-you will want the receipts.
Iceberg in Practice is an educational series for the people actually running Apache Iceberg: data engineers and architects who have tables in production, or are about to.
Plenty has been written about what Iceberg is. Much less about how to run it well: what the REST catalog actually standardizes, how to point four engines at one table without breaking anything, which catalog decision you'll still be living with in five years, and the unglamorous maintenance work that bites at scale.
That's this series. Each piece stands alone, teaches something you can use this week, and builds toward a complete picture of a well-run open lakehouse: real configs, real tradeoffs, real production lessons.
Next up: Iceberg table maintenance in production, the compaction and snapshot-expiry work that stops being optional once multiple writers share your tables.
Apache and the names of Apache projects referenced here are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other trademarks, product names, and company names are the property of their respective owners.