Databricks logoAgents

What an Agent Should Be Allowed to Assume

Hand an AI agent a table name and a task, and before it writes a line it has made a dozen assumptions. That `customer_id` identifies a customer and is unique. That `status` has a small set of values and `active` is one. That `amount` is a number, in one currency, never null. That the `orders` it found is *the* `orders`. That today's shape is the shape it was told about.

What an Agent Should Be Allowed to Assume

Hand an AI agent a table name and a task, and before it writes a line it has made a dozen assumptions. That customer_id identifies a customer and is unique. That status has a small set of values and active is one. That amount is a number, in one currency, never null. That the orders it found is the orders. That today's shape is the shape it was told about.

Every one is a guess, and the output is fluent enough that the guesses are invisible. In a lakehouse the failure mode is not a crash — it is a query that runs, returns rows, and is wrong in the same places a hurried human would have been wrong.

The fix is not a larger model. It is a shorter list of things the agent is allowed to assume. Here is what our Databricks Unity Catalog integration takes off that list.

Assumptions converted into facts

Import a Unity Catalog estate into CoreModels and those guesses become queryable, governed facts.

Identity is exact. Every dataset is addressed as catalog.schema.table — no ambiguity about which orders is meant, and no fuzzy name matching in the middle of an agent's reasoning.

Types are verbatim. Each column's native full_data_type is preserved exactly as Unity Catalog holds it, not rounded to "a number" by the time an agent reads it. A decimal with declared scale and a floating-point double are the difference between a correct financial total and a plausible one.

Nullability is recorded. A column declared NO for nullable carries an explicit not-null check, so "can this be missing?" is a lookup, not an inference.

Keys are honest, including when that is inconvenient. Primary-key rows are grouped by their constraint. A single-column key marks the column unique; a composite key marks its members not-null and records a composite-key marker on the table — it never tells an agent each member is individually unique. That refusal deletes a whole family of silently fanned-out joins.

Relationships are declared, not inferred. Foreign-key rows become governed references, so an agent joining two tables follows an assertion somebody made, not a naming coincidence.

Allowed values are governed. Where a taxonomy exists, the value set is a fact an agent can enumerate instead of a set it can hallucinate.

Lineage has direction. Rows from the system lineage table become dependency edges from the produced table to what it reads, with out-of-extract sources flagged external. "What breaks if this changes?" becomes a traversal.

Consultation, not scraping

Agents reach all of it over the Model Context Protocol. CoreModels serves an MCP endpoint at /mcp, secured with OAuth — modern clients handle discovery and the authorization flow — and the tools that matter for lakehouse grounding are read-only, at Viewer role.

An agent orients with get_vendor_integration_status: which connectors are registered, what artifacts each expects, and for databricks, whether this project has an imported estate at all, when, with what fingerprint, and how many datasets are governed. Then it walks the governed structure — types, elements, taxonomies, references, lineage — through the project read tools.

The interesting one is audit_vendor_project. Given a fresh information_schema extract — inline as artifacts, or by URL as artifactUrls when it is too large to inline — it compares the estate against the governed model and returns machine-shaped results: counts, and findings each carrying a section, a severity, a stable code, an exact subject, and a detail.

That shape is the part built for agents. A drift finding is not a paragraph to paraphrase; it is field-type-drift on main.sales.orders.order_total with the governed type and the artifact type side by side. Agents can branch on codes. They cannot reliably branch on prose.

The freshness question nobody used to ask

Because the audit is read-only and cheap, an agent can do something most human engineers never bother with: check that its ground truth is still true before acting.

An agent about to generate a pipeline, a migration or a report can first establish that the tables it depends on have not drifted — and if they have, say so, name the column, and stop. A human might do that once a quarter. An agent can do it on every task, for one read-only call.

Undocumented tables are now agent debt

One conformance rule deserves calling out. When a table or view has no comment, the audit raises table-no-comment, and the message says why: undocumented lakehouse assets resist governance and agent grounding.

Every undocumented table is a table your agents will improvise about, confidently. The audit turns invisible documentation debt into a countable finding with an exact subject — and because governed descriptions flow back out (generated Delta DDL carries descriptions and governed allowed values on column comments), writing one description pays out everywhere a reader looks, human or otherwise.

Lossiness is an uncertainty channel

One more thing a grounded agent gets that a scraping agent never does: a statement of what could not be represented.

Lossiness records travel beside successful results — a kind, a path, an explanation. A type with no exact slot, approximated. A constraint that could not be enforced, relaxed. A view skipped from generated DDL because views are derived objects. An estate too large for its snapshot to be stored.

An agent reading those records knows where the model is confident and where it is not — a far better input than a source that answers everything with identical certainty. Calibration is hard to teach and easy to hand over.

Where an agent's authority ends

Precise semantics are not a free hand, and the boundary is structural rather than aspirational.

Everything on the public MCP endpoint is read-only. The tool that writes estate structure into the governed model, import_vendor_project, requires the Admin project role and is served only on the separate admin endpoint — and even then the import is additive, so it cannot overwrite what stewards have governed. An agent can detect drift, explain it, and draft what would resolve it. Deciding what a column means stays a human act with a role check in front of it.

There is a useful middle register: generate_vendor_artifacts lets an agent produce the governed model's Delta DDL as a reviewable artifact — a draft carrying governed truth into a pull request, where people do what people should.

That division of labour is the whole proposition. Agents supply evidence and drafts at machine speed; humans keep authority over meaning; and the list of things an agent may assume gets shorter every time somebody documents a table.

The Databricks Unity Catalog quickstart in the CoreModels docs covers the import and the MCP tool calls agents use.