Transcription Is Where Conformance Goes to Die

Open almost any codebase that claims to follow an open data standard and, sooner or later, you find the file. It sits in a schemas/ folder, named something like customer.schema.json, and near the top there is a comment: adapted from the MACH ODM entity definition. No version. No date. No link back to the source. Somebody, once, working against a deadline, read a published entity document and typed out what they believed it said.

That file is the problem in miniature: the point where a standard stops being a standard and becomes one team's recollection of one.

The gap that produces the file

The MACH Alliance publishes its Open Data Model as documentation — entity documents written in Markdown, in the open, in a public standards repository. They are good documents. Each opens with a purpose paragraph, carries a field table with a practice column showing which fields a conforming implementation MUST provide and which it merely SHOULD or COULD, and includes a YAML Schema Definition section with OpenAPI-style schema blocks plus a realistic sample object.

For a human trying to understand what a customer or an order means, that format is close to ideal. For every machine you own, it is unreadable. Your API gateway cannot enforce a fenced code block. Your validator has no opinion about Markdown tables. Your warehouse does not know what a practice level is. Between the standard as published and the standard as enforced sits an unglamorous manual step, and that step is where conformance quietly dies.

Five ways the manual step goes wrong

We built ODM support because the same five failures kept appearing in that gap.

The copy has no cord back to the original. The moment the YAML is pasted into a repository file, there are two versions of the truth and no relationship between them. The standard is revised; the transcription does not follow, because nothing in it knows it is a transcription. A year later, the only honest answer to "is this still aligned?" is "someone would have to read both and compare."

The cross-references dangle. ODM's schema blocks reference each other the way OpenAPI documents do, through #/components/schemas/... pointers — sensible inside the entity document. Lift a block out into a standalone schema file and every one of those pointers aims at a location that does not exist. Some validators warn immediately; others resolve lazily and stay silent until a payload finally touches the broken branch in production.

Quoting is meaning, and quoting is what gets destroyed. This is the nastiest one. Suppose the document defines a status field as a string whose allowed values are "1", "2", and "3" — quoted deliberately, because they are codes, not counts. Run that YAML through a casual loader and the quotes evaporate: the values re-type as numbers. You have transcribed a schema no valid instance can satisfy — every string payload fails the enum, every numeric payload fails the type — and the diff looks perfectly faithful in review.

The semantics stay behind. Most of what makes an entity document valuable is not inside the YAML at all. The practice levels are in a Markdown table, the purpose is in prose, the known-good instance is in its own section. Hand transcription keeps the structure and abandons the guidance, because a JSON Schema file offers no obvious place for any of it. What survives is the shape of the standard with its intent removed.

Batch work fails without a record. The Open Data Model is not one entity; it is a growing set of them, and converting a catalog by hand means dozens of documents processed by someone who is bored by document eleven. When one file is malformed, a human either stops the whole job or skips it and forgets to say so. Neither leaves anything you can audit.

What we built instead

CoreModels, our schema governance platform at ARAMAI, treats a MACH ODM entity document as a first-class schema format — format key odm, one of the fourteen our transform surface speaks. You post the Markdown exactly as published to POST /graph/transform/odm/convert/{projectId} on a host such as https://coremodels.example.com, and get back a formal JSON Schema 2020-12 document. The call is stateless and Viewer access is enough, because converting a public document changes nothing.

Each failure above has a specific answer in that conversion. The entity's own schema becomes the document root, supporting schemas move under $defs, and every #/components/schemas/... pointer is rewired to match — a self-reference collapses to #. Where a pointer genuinely cannot resolve, it is preserved verbatim and reported rather than silently repaired.

The translator is a structural rewriter, not a whitelist: types, required lists, patterns, enums, bounds, defaults, and keywords it has never seen are carried across verbatim, and only the structure is rearranged. Scalar quoting survives, because the pipeline reads the YAML's representation rather than eagerly deserializing it — a deliberately quoted "42" stays a string, and only plain scalars re-type.

The prose comes along too. The entity-purpose paragraph becomes the description; the practice column lands on each property as an x-odm-practice annotation, an extension keyword validators ignore and your own tooling can read; the first sample object attaches as examples; and the root is stamped with x-odm-source, naming the standards repository the document came from.

Batches are first-class. Send several entity documents in one call and the response is a bundle keyed by kebab-cased entity name. Files without a YAML Schema Definition section are skipped as what they are — not entity documents — and a file that fails to convert becomes a named structural-drop entry in the lossiness ledger while the rest of the bundle proceeds.

And when a document is not what it claims to be, the converter says so instead of guessing: a missing schema section, a block that is not valid YAML, a section defining no named schemas — each fails with an error naming the problem.

The file you get to delete

None of this is exotic engineering. It is the unglamorous work of turning a document convention into a deterministic function, so that "does our schema match the standard?" stops being an archaeology question and becomes a command you can run.

The measure of success is that stale file in schemas/. Once conversion is a pipeline step rather than an afternoon someone spent, the file with the apologetic comment has no reason to exist.

For the endpoint reference, see the schema transform documentation in the CoreModels docs.