The Field Number Nobody Wrote Down

A team retires a field. It has been dead for a year, nothing reads it, and the pull request that removes it is three lines long and gets approved in under a minute. Months later somebody adds a new field and takes the next number that looks free — the same one. Both changes were correct on their own. Together they mean an older client and a newer one now disagree about what a stretch of bytes says, and the disagreement surfaces as a value that is quietly wrong somewhere nobody is watching.

Protocol Buffers has a defense against exactly this. The reserved keyword exists so a retired number and name can be fenced off permanently. But it only works while the schema's history stays inside the file, and the moment the meaning of that schema has to live somewhere else — it always does — the history stops traveling with it.

That gap is why we built Protocol Buffers support into the CoreModels transform surface.

What proto3 knows that nothing downstream can say

Proto3 is unusually precise about facts most schema languages have no vocabulary for. It distinguishes fifteen scalar types where a warehouse column sees one or two: uint64, sint32, and fixed64 are all "a number" to a table, but on the wire they are different encodings with different signedness and size behavior. It distinguishes implicit presence — a plain scalar always carries a value — from tracked presence, which is what an explicit optional label and a message-typed field give you. It carries field numbers, which are not metadata but the identity of a field on the wire. It has oneof — these fields are mutually exclusive — a constraint most targets cannot express. It has no date-time scalar, so the ecosystem converged on the well-known google.protobuf.Timestamp message instead.

Every one of those facts is load-bearing. Almost none survive contact with the next artifact downstream.

The retyping is where it dies

Analytics asks for the same concept as tables. Someone opens the .proto and writes DDL by hand. Names come across, rough types come across, and then the losses start — none of which raise an error.

uint64 becomes an integer column and the unsignedness is simply gone — not overridden, not flagged, just absent. bytes becomes a string. The payment group that proto3 declared mutually exclusive becomes two nullable columns that can both be populated at once. An enum becomes a free-text column that accepts anything. The field numbers do not appear at all, because DDL has nowhere to put them — which would be fine, except that they were not written down anywhere else either. The result reviews cleanly: plausible, and slightly wrong in four places.

The trip in the other direction is worse

Plenty of teams model first — in a governance tool, in JSON Schema, in a data contract — and then need a .proto for the service layer. So someone hand-writes one, minting field numbers as they go.

Hand-authoring has no memory. It does not know which numbers were assigned in an earlier generation, which were retired and fenced, or that the specification reserves the 19000–19999 range for the protocol's own implementation. It produces a file that compiles — the least demanding test a wire contract can pass.

Everyone re-derives; nobody reconciles

Now let the calendar run. The .proto changes because the service changed, the DDL because analytics needed a column, the partner-facing JSON Schema because someone rewrote a description. Within two releases, three documents describe one concept and none is derived from the others. Ask which is right and there is no procedure that answers — only seniority. No linter closes that gap, because each schema is individually valid. The problem is structural: there was never a single place where the meaning lived and the formats were projections of it.

What the pain is actually asking for

Not another converter — a change of status. The .proto should stop being the truth and become one faithful projection of a governed definition, on the condition that the projection keeps the facts that make it a wire contract. A translation that drops field numbers has not translated the schema; it has paraphrased it.

That condition shapes how we handle the format. Proto3 is a first-class format on the transform surface under the key protobuf (alias proto), in both directions, scoped to a single .proto file. Import one and its messages become Types, its fields become Elements with their wire spelling preserved, and its enums become Taxonomies whose terms keep their numeric values. Nested messages become child types that remember which message enclosed them. repeated becomes a collection. google.protobuf.Timestamp becomes a real date-time. Plain scalar and enum fields decode as required because proto3 gives them implicit presence, while explicitly optional and message-typed fields do not.

The wire-level facts ride along in the transform's annotation channel: the package, import clauses, options, reserved statements, field numbers, verbatim scalar tokens, field options, oneof group names, and a map's key and value types. Export back to proto3 and they re-emit exactly. Elements that never had a number are minted the next free one in declaration order, skipping numbers already in use and the implementation-reserved range — so growth is wire-safe by construction rather than by care.

And where it does not fit, we say so

Every translation returns a lossiness ledger: a kind, a path, and a plain-English explanation for anything that could not be carried exactly. A uint64 reported as a type approximation, with the original token kept so the export is not a guess. A flattened oneof reported as a semantic narrowing, named after the group. A service block reported as a structural drop, because RPC definitions are not data schema. A type from an unresolved import approximated with its name preserved.

We are equally blunt about boundaries. Proto2 is declined outright rather than half-parsed — a declared syntax = "proto2" or a required field is a refusal, not a warning. A file with no syntax statement defaults to proto2 per the specification; we decode it with proto3 semantics and put that assumption in the report.

The number nobody wrote down is the small version of a general problem: the most important facts in a proto3 file are the ones least likely to survive being retyped by a human under deadline. Put them somewhere they can be governed, and the retyping stops being necessary.

To try it against a file of your own, start with the schema transform quickstart in the CoreModels docs.