Good Neighbors: proto3 Among the Formats It Has to Live With

Protocol Buffers was designed for a conversation between two running programs. Everything distinctive about it follows from that: the compactness, the field numbers, the fanatical attention to what happens when one side upgrades before the other. It is a contract about bytes in flight, and it is very good at being one.

Data estates are not conversations between two programs. They are arguments among a dozen tools that each hold a partial description of the same business object, and the interesting failures happen at the seams. So the question worth asking about proto3 is not "is it a good schema language" — it is — but "what happens at its edges, and who is responsible for the translation."

The neighborhood

Each of proto3's neighbors owns a slice of the truth and is unapologetic about it. Avro dominates data in motion and carries logical types like timestamps natively. JSON Schema owns validation at API boundaries and in the document world. SQL DDL is the language of the warehouse. LinkML and OWL carry semantics and ontology bindings the wire formats have no vocabulary for. Data-contract formats such as the Bitol Open Data Contract Standard wrap schema in operational agreements. Apache Ossie semantic models describe measures and dimensions for the analytics layer. Schema registries mediate between producers and consumers on streaming platforms, holding several of these types side by side.

Nobody is replacing anybody. Real organizations run four or five at once, and the same order concept ends up expressed in most of them.

Proto3's distinctive contribution to that crowd is wire discipline. It says things the others cannot: field numbers as permanent identity, fifteen scalar types that distinguish encodings rather than just value ranges, implicit versus tracked presence, reserved fences against number reuse, and oneof as a mutual-exclusion constraint. Conversely it lacks things its neighbors consider unremarkable: no inheritance, no date-time scalar — hence the well-known Timestamp message everyone imports — no constraint language for lengths or bounds, no first-class relationship between entities, and no concept of a field that belongs to no message.

A translation layer that respects proto3 must carry the first list faithfully and be candid about the second. Most tools do the opposite: they carry what is convenient and stay quiet about the rest.

Hub, not pairwise

The structural choice that makes this tractable is refusing to build converters in pairs.

Pairwise translation looks cheap until you count. Every new format needs a translation to every existing one, each pair invents its own opinions about presence and enums, and the opinions disagree. Instead, every format meets in one shared intermediate representation. Proto3 is decoded once — under the format key protobuf, with proto as an alias, in both directions, scoped to a single .proto file — and from there it can be encoded to anything else the surface speaks: JSON Schema, Avro, JSON-LD, ShEx, SQL DDL, OWL in Turtle, LinkML, Ossie semantic models in YAML or JSON, ODCS contracts, and the Synapse draft-07 JSON Schema profile, with a couple of deliberate one-way cases where a two-way claim would be dishonest.

The payoff is that every hop accounts for itself the same way. Proto3 into Avro and proto3 into JSON Schema are both exercised directly by our tests, and both return the same kind of lossiness ledger. What only proto3 can say — verbatim scalar tokens, field numbers, reserved statements, options — rides along in dedicated annotations so a round trip restores it exactly. What proto3 cannot say is reported on export rather than approximated in silence.

Composing with a schema registry

Format translation is not the whole story. Streaming estates govern their schemas through a registry, and our Confluent connector audits an exported set of registry subjects against the governed model.

Here honesty is more useful than coverage. Avro subjects — the registry default — parse fully. Protobuf subjects are counted and reported as an informational coverage finding rather than parsed, which says plainly that a part of the estate is not governed through that path. It does not fail the audit, and it does not pretend. To govern a proto3 schema, the transform surface decodes the .proto itself; the registry audit tells you which subjects it did not reach. We would rather ship a boundary you can see than a claim you have to test in production.

Why neutrality is a technical position, not a slogan

The position we refuse to take is that any one format is the real schema and the others are exports of convenience.

Watch what happens when a tool quietly picks a favorite. If it privileges JSON Schema, field numbers and presence semantics degrade into annotations nobody maintains, and the wire contract stops being trustworthy. If it privileges protobuf, ontology bindings and constraint fidelity evaporate, and the semantic layer becomes decorative. Either way one community's precision becomes another's noise, and the losses are invisible because the tool never mentioned them.

Neutrality means the governed model sits at the center and every format — proto3 very much included — is a projection with known, recorded losses.

It also means we do not fork or extend the formats we speak. We implement the published proto3 language specification, and what we emit is plain, deterministic proto3 that any standard toolchain compiles. We do not encode CoreModels concepts into custom options or comment conventions, because a file that only round-trips through our own tooling is not interoperability — it is a dialect. When the model holds something proto3 cannot express, the export says so in its ledger and emits valid proto3 anyway.

Neutrality is also why scope honesty matters. The coder handles a single proto3 file; it does not resolve import statements into other files, so imported type names are preserved verbatim, approximated openly, and re-emitted intact. Proto2 is declined rather than half-parsed. Pretending otherwise would buy a better demo and a worse contract.

The practical upshot

Treat your .proto files as what they are: the wire community's authoritative voice in a conversation that also includes the warehouse, the validators, the registry, and the semantic layer. A neutral hub lets each voice keep its accent — the service team keeps its numbers and presence rules, analytics gets DDL it can read, the ontology work survives in formats that can hold it — while one governed model keeps them saying the same thing, with a written record of what each dialect could not express.

To see the full format list and try a proto3 round trip, start with the schema transform quickstart in the CoreModels docs.