Namespace design, the grant model, ownership, and the governance decisions that are architecture

Production Databricks Notes: architecture decisions, investigations, implementation patterns, and lessons for data and AI systems that must operate beyond the demo.

Unity Catalog is usually introduced as access control, which is accurate and undersells it in a way that leads teams to treat it as configuration.

The catalog structure is where every other decision in this series lands. The reliability contract from Week 1 describes guarantees; the catalog is where those guarantees become enforceable or stay aspirational. The identity separation from Week 3 has no effect until there are grants keyed to those identities. The tenant boundary from the companion note is a catalog design question before it is a security question.

This article is about designing that structure deliberately.

The grant model, precisely

Three-level namespace: catalog.schema.table. Privileges are granted on securable objects and can be granted at a container level to apply to current and future objects inside it.

The rule that surprises people, stated verbatim in the documentation:

To read from a table, a user needs SELECT on the table, USE CATALOG on the parent catalog, and USE SCHEMA on the parent schema.

All three. This is the source of most "I granted SELECT and it still says permission denied" support questions, and it is worth understanding as a design feature rather than a hurdle.

In practice that means three statements, not one:

GRANT USE CATALOG ON CATALOG prod            TO `analysts`;
GRANT USE SCHEMA  ON SCHEMA  prod.sales      TO `analysts`;
GRANT SELECT      ON TABLE   prod.sales.orders TO `analysts`;

Grant the container privileges once per group and forget them; the per-object grants are the ones that change. Granting at the container level applies to current and future objects inside it, which is the behaviour you want for USE, and the behaviour to think twice about for SELECT:

-- every table in the schema, including ones nobody has created yet
GRANT SELECT ON SCHEMA prod.sales TO `analysts`;

That line is convenient and it is also a standing decision that the next table anyone creates in prod.sales is readable by analysts. Sometimes correct. Worth being a decision rather than a side effect.

Checking what a principal holds, which is the query you want during an incident rather than after it:

SHOW GRANTS `analysts` ON CATALOG prod;
SHOW GRANTS ON TABLE prod.sales.orders;

The design consequence is that USE CATALOG is a real boundary. Without it, nothing inside the catalog is reachable regardless of what else you granted. That is what makes catalog-per-tenant a structural boundary rather than a naming convention, and it is the property that makes the isolation argument in the companion note work.

Granting authority follows ownership: only catalog owners or principals with MANAGE on the catalog can grant USE CATALOG, and the same shape applies at the schema level.

The full privilege list is long, over forty types, including BROWSE, EXECUTE, MODIFY, READ VOLUME, CREATE MODEL, APPLY TAG, and MANAGE. You will use a handful. The others exist, and the moment to look them up is when you are designing a role rather than when you are debugging a denial.

Design the namespace around blast radius

The most common structure is environment.domain.table or domain.environment.table, and teams pick between them by preference. The choice has a consequence worth making explicit.

Environment at the catalog level: prod.sales.orders, dev.sales.orders.

USE CATALOG becomes an environment boundary. A principal without USE CATALOG on prod cannot reach anything in production, whatever else they hold. Environment separation is enforced by the strongest boundary available, in one grant.

The cost: a domain's objects are split across catalogs, so anything reasoning about a domain across environments crosses catalogs.

Domain at the catalog level: sales.prod.orders, sales.dev.orders.

Domain ownership aligns with the catalog owner, which matches how teams are usually organised. The cost is that environment separation is now a schema-level boundary, and schema-level boundaries are crossed by a mistaken USE statement.

I would put environment at the catalog level in almost every case. Domain alignment is an organisational convenience; environment separation is a correctness property. When they conflict, correctness wins, and the domain-ownership problem has other solutions (ownership at the schema level, tags, a naming convention) while a weak production boundary does not.

flowchart TB
    R["Put the boundary you least want crossed<br/>at the CATALOG level.<br/>Crossing it requires an explicit grant."]

    R --> A
    R --> B

    subgraph A["Environment at catalog level (recommended)"]
        direction TB
        A1["prod.sales.orders<br/>dev.sales.orders"]
        A2["USE CATALOG is an environment boundary.<br/>No grant on prod means no access to production,<br/>whatever else the principal holds."]
        A3["Cost: a domain spans catalogs."]
        A1 --- A2 --- A3
    end

    subgraph B["Domain at catalog level"]
        direction TB
        B1["sales.prod.orders<br/>sales.dev.orders"]
        B2["Catalog owner matches team ownership."]
        B3["Cost: environment separation drops to<br/>schema level, crossed by a stray USE."]
        B1 --- B2 --- B3
    end

Alt text: A comparison of two namespace layouts. Environment at catalog level, recommended, uses prod.sales.orders and dev.sales.orders, making USE CATALOG an environment boundary so that no grant on prod means no access to production whatever else the principal holds; its cost is that a domain spans catalogs. Domain at catalog level uses sales.prod.orders and sales.dev.orders, aligning catalog ownership with team ownership; its cost is that environment separation drops to schema level, where a stray USE statement crosses it.

The general rule: put the boundary you least want crossed at the catalog level, because that is the level where crossing requires an explicit grant somebody had to write.

Ownership is an operational decision

Every securable has an owner, and the owner can grant. It is tempting to leave ownership wherever it landed, which is usually whoever ran the CREATE statement.

That reproduces the identity problem from Week 3 one layer up. A catalog owned by a person means the granting authority for that catalog leaves when they do, and you discover it when somebody needs access during an incident.

Own catalogs with groups or service principals. The person who created the object should not be the long-term owner of it, for the same reason the person who wrote a job should not be the identity that runs it.

ALTER CATALOG prod OWNER TO `data-platform-owners`;
ALTER SCHEMA  prod.sales OWNER TO `sales-data-owners`;

This also makes ownership reviewable, and the review is a query rather than an audit. The system tables carry it:

-- Catalogs owned by a person rather than a group or service principal.
-- Anything this returns is a finding.
SELECT catalog_name, catalog_owner
FROM   information_schema.catalogs
WHERE  catalog_owner LIKE '%@%';

The LIKE '%@%' test is crude and works, because human principals are email addresses and groups are not. Run it on a schedule. It is cheap to check and expensive to discover reactively, which is the profile of every control worth automating.

One scoping caveat: information_schema.catalogs shows catalogs bound to the current workspace that the querying principal has permission to interact with. It is a workspace-level view, not an account-wide one, so a multi-workspace estate needs this run per workspace or replaced with an account-level query. Check the current reference for what is available at account scope before building a compliance report on it.

The same shape works one level down against information_schema.schemata, and the grants themselves are queryable through information_schema.table_privileges when you need to answer "who can read this" without clicking through the UI.

What the catalog gives you for free, and what it does not

Lineage is captured automatically for supported operations. It is useful and it has a limit worth knowing: it records what the platform observed. Work that leaves the platform is not in it, so an export to a file, a transformation in an external application, or a copy someone made is invisible to it. Lineage answers "what does this table depend on inside Databricks", which is a narrower question than "where did this number come from".

Audit logging records access. Its value depends entirely on the identity model. If workloads run as their authors, the audit log cannot distinguish a human query from a scheduled job, and the forensic value drops to near zero at the moment you need it.

Week 3 covered why a workload running as its author destroys the audit log's forensic value. Unity Catalog inherits that problem exactly, and adds one: a grant made by a human on behalf of a service is indistinguishable from a human granting themselves access.

Tags support classification and can drive policy. Worth applying at creation time through the bundle definition rather than retrospectively, because a retrospective tagging exercise over an existing estate is a project rather than a task.

Governance anti-patterns

Six that recur, roughly ordered by how much trouble they cause.

Grants to individual users rather than groups. Every joiner and leaver becomes a manual change, and the drift is invisible because nothing fails when a grant is left behind. Grant to groups exclusively; the exception you are about to make for a single service principal should be a group with one member.

Ownership by individuals. Covered above. The cheapest of these to fix and the one most likely to block something urgent.

ALL PRIVILEGES as a shortcut during setup. It is never revisited. If you need it temporarily, put an expiry in the ticket, because nothing in the platform will remind you.

Environment separated only by naming convention. prod_orders and dev_orders in one schema is a comment, not a boundary. This is the single most common structural weakness I encounter.

Catalog structure that mirrors the org chart. Reorganisations happen more often than data architectures should change. Structure by domain and environment, which are properties of the data, rather than by team, which is a property of this year.

Permissions granted outside the bundle. A grant applied by clicking exists only in the workspace. Nothing in the repository describes who can read production, so nothing can detect that it changed. Resource definitions in Declarative Automation Bundles cover permissions; use them, and make drift detectable.

Governance in the bundle

The Week 3 argument applies here without modification. Permissions are part of the deployable system.

The permissions mapping takes a level plus exactly one of group_name, user_name, or service_principal_name. Allowed levels at the top level are CAN_VIEW, CAN_MANAGE, and CAN_RUN; job-specific levels such as CAN_MANAGE_RUN belong under the job resource, because a top-level block applies to every resource in the bundle including pipelines that have no such level:

permissions:
  - level: CAN_VIEW
    group_name: data-analysts
  - level: CAN_RUN
    group_name: data-engineers
  - level: CAN_MANAGE
    service_principal_name: sp-data-platform-prod

targets:
  prod:
    mode: production
    run_as:
      service_principal_name: sp-data-platform-prod

Note that user_name is available and should be the exception you have to justify in review, for the same reason grants to individuals are an anti-pattern. A bundle that names a person is a bundle that breaks when that person leaves.

Defining grants in bundle resources gives you three things: the deployed permission state is describable by reading the repository, permission changes go through the same review as code changes, and databricks bundle plan on a schedule surfaces drift between the grants you declared and the grants that exist.

The organisational half is harder than the technical half. Emergency grants happen and are usually correct. The discipline is bringing them back into the repository afterwards rather than reverting them, and that only happens if somebody owns it.

A review checklist

Ten questions. Unlike the Week 1 set, most of these have a query that answers them, so an unanswered one is usually a gap in your tooling rather than in your thinking.

Which boundary is at the catalog level, and is it the one you least want crossed?
Are any catalogs or schemas owned by individual users?
Are there grants to individual users rather than groups?
Does any principal hold ALL PRIVILEGES on a production catalog?
Is environment separation structural, or a naming convention?
Do production workloads run as service principals?
Are permissions defined in the bundle, or applied by hand?
If row filters are used, does a test assert every table in the catalog has one?
Does any table have both a row filter and a rollback plan that depends on time travel? (These do not compose; see the companion note.)
When was the last time somebody reviewed who can read the production catalog?

Questions 2, 5, and 6 are the ones I would fix first. All three are cheap, and each one unblocks something else on the list.

The connection back

Every article in this series has produced a decision that had to be written down and owned. This one produces the place those decisions become enforceable.

A reliability contract that says "only the ingestion service writes to bronze" is a sentence until there is a grant that makes it true. A deployment identity separate from a workload identity does nothing until they hold different privileges. A tenant boundary is a design until USE CATALOG enforces it.

Governance is where the guarantees you wrote down either become properties of the system or stay documentation.

Companion to the note on tenant isolation options in Unity Catalog. Privilege model and access requirements verified against Databricks documentation on 2026-07-29. The namespace recommendations and anti-pattern list are engineering judgement rather than a Databricks platform guarantee.