Protocol Buffers logoOutcomes

What Your Service Team Stops Being Asked

Every team that owns a `.proto` file also owns an unpaid second job: being the interpreter for it. The requests arrive individually, each one reasonable, each one costing an afternoon. Can we get the order model as tables? The partner needs this as JSON Schema by Thursday. Which of these fields are actually required? Can you add a discount field without breaking the mobile clients?

What Your Service Team Stops Being Asked

Every team that owns a .proto file also owns an unpaid second job: being the interpreter for it. The requests arrive individually, each one reasonable, each one costing an afternoon. Can we get the order model as tables? The partner needs this as JSON Schema by Thursday. Which of these fields are actually required? Can you add a discount field without breaking the mobile clients?

None of those is a hard question. They are hard scheduling problems, because the only people who can answer them are the two people whose calendars are already full.

Here is what changes once the proto3 schema is governed rather than merely stored — request by request.

"Can we get this as tables?"

Before: a meeting, a shared screen, and an analytics engineer transcribing a .proto she has never had reason to read. She asks what sint32 means for a column type. She guesses at the oneof. She produces DDL that is 90% right, and the 10% is invisible.

After: she exports it. The same governed model that the proto3 file was imported into emits SQL DDL directly, and the presence semantics travel with it — fields proto3 treats as always present arrive marked required, explicitly optional ones do not. The interpretation is not a conversation any more; it is a call. The service team's involvement drops to zero — the actual win, because their involvement was never adding information, only decoding it.

"The partner needs JSON Schema, and the pipeline wants Avro"

Two different consumers, one afternoon each, historically. Both are now exports of the same governed model: proto3 decoded once into a shared representation, then encoded into whichever format the consumer speaks. Our test suite exercises exactly those two paths — a proto3 schema transiting the shared representation into Avro and into JSON Schema — because a claim like that should be held down by a test rather than a diagram.

What matters is not that the conversion is fast. It is that both artifacts are derived from the same definition, so when the order model changes next quarter, they change together instead of drifting apart.

"Which fields are actually required?"

This one used to have no honest answer at all, because "required" means three different things in three places, and proto3's version of it is subtle enough that even service engineers hedge.

After import, it is browsable. Each message is a Type, each field an Element carrying its own presence answer, each enum a Taxonomy whose terms keep their order and wire spelling — which means the governance lead can read the model without reading proto3, and the answer she reads is the one the wire enforces. Nested messages appear as their own types, every field's wire spelling kept as its label.

"Add a field. Don't break anything."

The steward adds the element in CoreModels, where the discussion about what it means belongs. Then the service team exports the project back to proto3 — and this is the step worth understanding precisely, because it has a boundary we would rather state than have you discover.

The wire-level facts a .proto carries — field numbers, verbatim scalar tokens, field options, optional labels, oneof groups, a map's key and value types, the package, import clauses, and reserved statements — ride in the transform's annotation channel, which a project's schema store does not persist. On the stateless routes, where that channel lives, every field that came from a file re-emits exactly, and an element without a number is minted the next free one — skipping numbers already taken and the 19000–19999 range the specification reserves for the implementation. An export straight from the project mints all of its numbers fresh by that same deterministic rule: a valid, wire-safe contract that is schema-equivalent to the model, not a continuation of the original file's numbering. So treat the exported file as the wire contract from that point on — commit it, and the deterministic encoder keeps every later diff to exactly the change that was made.

Wire safety comes from how the numbers are assigned, not from someone remembering to check.

The quieter outcome: the export behaves like source

There is a second-order effect that teams notice in week two rather than week one. The encoder is deterministic: the same model produces byte-identical text every time, and a decode-encode cycle through the transform is a fixed point.

That is what makes the generated .proto fit in a pull request. The diff for a one-field change is one field. Nothing reshuffles, nothing reorders, no cosmetic churn buries the substance. A reviewer can apply the same judgment to it as to hand-written source, which means the generated artifact can live in the service repository instead of in a side channel that slowly loses trust.

What the after-state honestly does not include

Proto3 cannot hold everything a governed model can, and the outcome we promise includes knowing that precisely.

It has no inheritance, so a parent-child link between types is reported and dropped rather than faked. It has no free-standing relations. Its enum values are flat, so a hierarchy inside a taxonomy flattens. repeated carries no item bounds, so collection minimums and maximums relax. It does not distinguish null from absent, so a nullability flag has nowhere to go. It has no top-level fields, so an element belonging to no type cannot be represented at all. Each of those meets the encoder and produces a lossiness record — a kind, a path, an explanation — rather than a silent omission or a comment pretending to be a constraint.

The same applies on the way in. A flattened oneof, an approximated map, a bytes field, a skipped service block: all recorded, all located. A successful call with a non-empty lossiness list is the normal case, not a defect. success means it ran.

The shape of the change

Add up the four requests and the arithmetic is not really about hours saved. Before, four teams each held a partial copy of one concept and reconciled them by conversation. After, there is one governed definition and four generated projections, each with a written record of what its format could not hold.

The service team keeps everything it cared about — numbers, presence, reserved fences, exact scalar tokens. Everyone else stops needing an interpreter. And the question that used to be unanswerable, do the wire contract and the warehouse agree, becomes a report you can read instead of an argument you can lose.

If you want to run the loop yourself — import, export, and read the lossiness ledger — start with the schema transform quickstart in the CoreModels docs.