A Subject Is a Border Crossing

Two teams that will never attend the same meeting still have to agree on something, and in a Kafka estate that something is a subject in the Schema Registry. The producer owns a service and a release train; the consumer owns a different service, a different train, and different assumptions. Between them sits one versioned schema document that neither fully owns and both depend on.

Border crossings are where governance questions concentrate — so start with what this one handles, and what it was never built to handle.

What the ecosystem around the registry consumes

Draw the machinery and a clean division of labour appears. Brokers move bytes and know nothing about them. Serializers turn bytes into records, but only when they know the shape — which is why compact binary formats can travel as a schema id. The registry stores those schemas as versioned subjects — conventionally a key and a value subject per topic — and checks new versions for compatibility. Around that core sit the schema languages (Avro as the registry default, JSON Schema, Protobuf), the connectors feeding warehouses and lakes, the stream processors, and the CI pipelines where schema files are authored.

Every one of those layers consumes shape. None owns meaning: what a field is for, which enum values downstream logic may rely on, which record references which, and who agreed to any of it. That is not a criticism of the design — it is the design's boundary. And it says exactly where a governed model belongs: beside the registry, not inside it and not instead of it.

Composing, in the ecosystem's own vocabulary

Our connector sits alongside these pieces rather than between them. Its input is the registry's own vocabulary: an export of each subject's latest version — subject, version, schema id, schema type, schema — collected from the standard REST API by a loop you run yourself. Its output is the ecosystem's vocabulary too: plain .avsc Avro record schemas, ready for whatever registration workflow you run. Avro is an open Apache specification and generated schemas are ordinary instances of it — readable by any Avro tooling.

In between, the governed model keeps what the ecosystem cares about. Subjects become Types that remember their topic as a physical name and their key-or-value role, version, and schema id as metadata, so the governed view lines up with the registry view instead of floating free of it. Avro's optionality semantics survive the trip, enums become governed taxonomies, and named-record references are resolved across subjects — making explicit the graph that was always implicit in your registry.

The audit slots in at the ecosystem's natural checkpoint — continuous integration — complementing the registry's compatibility check with a semantic one at the same gate.

Your streaming estate does not end at the registry

Here is what a registry-only view misses. Order events on a topic land in a warehouse table. The customer record embedded in a payment event has a sibling in a CRM object. The pipelines moving all of it carry their own definitions. Meaning crosses these borders constantly; tooling almost never does.

CoreModels treats Confluent Schema Registry as one connector among thirteen — warehouses, catalogs, transformation frameworks, orchestrators, graph databases, SaaS platforms — that all parse into the same neutral estate model and expose the same verbs: additive import, read-only audit, artifact generation where it makes sense. Vendor detail rides as metadata on governed nodes rather than becoming new top-level concepts, so adding a connector never reshapes the model your governance depends on.

Because every estate lands in one model, cross-estate questions become expressible: where two systems describe the same physical relation, reconciliation links the pair with reciprocal same-as assertions. Streaming and warehouse objects rarely share a physical name, so what matters more here is that a subject and a table now live in one vocabulary, under one audit engine, with one drift loop, rather than in two tools that cannot see each other.

Meaning outlives its serialization

The neutral core has a second consequence. Because the governed model isn't registry-shaped, a governed Type can leave in more than one direction: JSON Schema for an API team, SQL DDL for a database, Protobuf for a gRPC surface, LinkML for a modeling workflow, OWL or ShEx for semantic tooling, an ODCS data contract, an Apache Ossie semantic model — alongside the Avro that goes back to the registry. We implement those published specifications rather than inventing private ones.

It works the other way too: generation emits one Avro record schema per governed Type, so a definition that entered from a warehouse can leave as a registry-ready record. The estate boundary stops being the modeling boundary.

Honesty is part of neutrality

Where the registry's world is richer than what this connector parses, we say so on every run. Protobuf subjects are counted and inventoried, not modeled, and each audit repeats that as a coverage finding with the count attached. The gap is specific and named — the platform's schema-transformation surface reads Protobuf as a first-class format, so the limitation belongs to this connector's parser and is stated as such, not blurred into vague "partial support."

A neutral model that overclaims its coverage is worse than a narrow one that tells the truth.

Why neutrality is load-bearing

It would have been easier to build a registry-specific tool: mirror the subjects, decorate them with descriptions, call it governance. Three reasons.

Vendor-shaped governance inherits every limitation of the vendor's model — you can only govern what that vendor happened to model. It creates lock-in in both directions; ours reads standard exports, emits standard Avro, and exports to open formats. And it has a shelf life: schemas outlive brokers, meaning outlives schemas, and organizations change streaming platforms far more often than they change what an order is. Governance bound to one vendor's model gets rebuilt at every transition; governance held in a neutral model gets a new connector.

The completed map

The registry keeps guarding wire compatibility, exactly as designed. CI keeps reviewing schema changes, now with a semantic gate beside the syntactic one. Producers keep registering schemas — some now generated from agreed definitions rather than written from memory. And one neutral model holds what no layer of the ecosystem was built to hold: the meaning of the estate, connected across streaming, warehouse, and everything else touching the same data.

How the pieces connect in practice — the export recipe, both API surfaces, and the CI gate — is in the CoreModels Confluent Schema Registry quickstart in our docs.