Stop Guessing the Union Branch: Avro for AI Agents

An agent is asked to produce test records for an orders topic. It has seen ten sample messages, and it writes what those samples look like: a JSON object with an age of 30.

The field is typed ["null", "long"]. In Avro's JSON encoding, a union value is written as a one-key object naming the branch — the correct payload wraps the number under its branch name — and the agent's version fails to deserialize. It is a small bug with an expensive shape: the agent was not careless, it was uninformed. The rule was invisible in the samples, because a library that applies it correctly leaves no trace of having done so.

This is the characteristic failure mode of agents working with streaming data, and it generalizes badly.

Samples are inference; schemas are facts

Give an agent records and it will reconstruct a schema by statistics. Three things it cannot recover that way, all of which Avro states explicitly:

Optionality. A field that is present and non-null in every sample may still be a ["null", T] union. The agent calls it required, and produces a consumer that breaks the first time production disagrees.

The closed set. A field showing three values across ten samples may be an enum with seven symbols. The agent treats it as free text, and the validation it writes accepts anything.

Precision and meaning of numbers. A thirteen-digit integer is a long, or it is a timestamp-millis logical type, and the two demand different downstream handling. Width has the same problem: int and long look identical in a sample and are not identical in a schema, a warehouse column, or a serializer.

Avro's premise is that the schema, not the data, carries the truth. Handing an agent the data and asking it to rediscover that truth inverts the format's central idea.

Consultation, not reconstruction

The fix is not a better prompt. It is giving the agent somewhere authoritative to ask.

CoreModels exposes an MCP server at /mcp, secured with OAuth. The public endpoint serves read-only tools; write tools live only on the admin endpoint, and per-project role checks apply regardless of which endpoint a token was minted for. An agent on the read-only endpoint sees everything it is entitled to see and changes nothing.

For an agent working with Avro, the useful calls are few and specific. list_projects and get_project_summary establish what exists — the types, elements, and taxonomies in a project. search_nodes and get_mixins_and_relation_groups go a level deeper. export_avro returns the project's schema as an Avro .avsc JSON string, optionally scoped to a space or to particular type nodes, so an agent writing a producer can request the exact record schema instead of inferring one. transform_schema takes a schema in any supported format and returns it in another — avro is valid on both sides — along with the executed plan and the lossiness ledger. audit_vendor_project compares vendor artifacts, including a registry's subjects export, against the governed model and returns structured findings.

That list is deliberately unglamorous. The point is not that an agent can do something spectacular; it is that it can stop guessing.

What precision actually buys

An agent that reads a governed Avro schema instead of sampling gets the three facts above for free, plus the ones that only matter when you get them wrong.

It learns that ["null", T] is optionality, and which fields have it. It learns the full symbol set of every enum, because in the governed model an Avro enum is a Taxonomy — a named list, not an array of strings buried in a JSON file. It learns that a timestamp-millis field is a date-time whose underlying encoding is an integer, so its records carry epoch-milliseconds rather than an ISO string. It sees the namespace, the doc strings, and the field defaults, because those survive the round trip. And when it writes records, it applies the union-wrapping rule that started this article, because the rule is stated rather than inferred.

Where Avro has been annotated with links to published vocabularies — riding as custom attributes inside the .avsc — the agent gets those too. That is the difference between "there is a field called status" and "there is a field called status bound to a specific term with a specific definition."

The ledger is agent-grade context

Every transformation returns a lossiness list: each entry a kind, a path, and a plain-English explanation. That structure is as useful to a model as to a person.

An agent that reads the ledger can report accurately: this export narrowed a three-branch union at this path and dropped the alternatives; this one approximated a decimal and lost its precision; this one could not enforce an allowed-value list in the target. An agent that ignores it hands over a clean-looking artifact and lets a human find the compromises later. The information is there, in a shape a tool can parse, so the honest behavior is the easy one.

Boundaries an agent cannot argue with

Grounding without limits is a faster way to be wrong at scale, so the limits are structural.

Every mapping plan — authored, inferred, or model-proposed — passes the same validation gate before anything executes, and execution after the gate is deterministic. An AI proposal gets at most one repair attempt, and a failed repair is a rejection; the gate is never loosened to accommodate a model, and a proposal's self-reported confidence carries no weight with it.

That mode also requires Editor or Admin membership and a server-configured key, and it sends schema content to the Anthropic API server-side — declared on the tool rather than discovered in a log. Tools carry explicit annotations, including which are read-only and which reach outside the system, so a client can reason about a call before making it.

The same restraint applies to advisory features: ontology term suggestions are search results and nothing more. They never write; binding a term is a separate, explicit act on the admin endpoint. Suggesting is not deciding.

The pattern that works

Consult, transform, review, then write. The agent reads the governed schema rather than guessing it, runs the translation it needs, and reads back the plan and the ledger. It presents both to a person, with a plain statement of what was lost. A human with write authority decides.

That leaves agents doing what they are good at — reading a great deal of structure quickly and proposing precise work — while every act of consequence stays where it belongs.

To connect a client and try a grounded Avro translation, start with the MCP quickstart in the CoreModels docs.