Two Questions That Sound Alike: Where ShEx Sits, and Why We Take No Side

"What does this term mean?" and "what must a conforming record contain?" sound like one question. They are two, answered by different artifacts, and confusing them is why so many schema discussions go in circles.

The first is answered by a vocabulary: a published set of terms, each with an identifier anyone can resolve. The second is answered by a shape: a statement that around a node of this kind, these predicates appear, this many times, with these datatypes or permitted values, pointing at those other shapes.

ShEx answers the second question — and that is most of understanding where it fits.

The neighborhood ShEx grew up in

The RDF world makes one large bet: identify things with IRIs, so meaning survives crossing an organizational boundary. RDF is the data model, Turtle and JSON-LD the serializations people exchange, OWL the home of formal ontology, and public vocabularies the dictionaries thousands of independent systems resolve identically.

What that stack long lacked was an approachable way to state conformance — not what a term means, but what a well-formed record around it looks like. ShEx grew into that gap out of community practice rather than a standards-track recommendation, and its signature is the compact syntax, ShExC: terse enough that a shape, its constraints, and its inheritance fit on one screen and survive review by a human being.

Adoption clusters where federation is the norm: public knowledge bases publishing entity schemas, research data commons, consortia with many contributors, no shared database, and a need to agree on structure without agreeing on infrastructure.

What a shape composes with

A shape never travels alone. It sits on two other layers: the vocabulary saying what schema:name means, and the datatype layer saying what the literal value is. The IRIs are the joints.

Our decoder is built for that composition. It ships with the conventional prefixes bound — XML Schema datatypes, the RDF and RDFS namespaces, a common public vocabulary, and a CoreModels namespace for names with no external home — so a document using the usual short forms resolves even without a declaration. Anything the document declares wins, and the prefix map is preserved so the export re-emits it. On the way out, the encoder declares only the prefixes it used, in stable order, keeping generated ShExC tidy and diffable.

That layering is also why ShEx is unusually valuable inside a governed model. Every Type and Element in CoreModels can carry mappings to external standards — URIs stating what a thing means independently of its local name. Most formats need a bolted-on convention for those: a reserved keyword, a custom attribute, a comment. ShEx needs none, because its predicates and shape names already are IRIs, so decoding a shape file delivers the semantic layer for free.

And it keeps travelling. Export that model as LinkML and the meaning arrives natively as a class URI on the class and slot URIs on the attributes; as OWL, the shape's class equivalence is stated outright; as JSON Schema, the mapping rides as a machine-readable annotation on each property; as SQL, in the column comments. Formats with a native slot for meaning get it; the rest get an honest annotation.

One hub, many spokes

Inside CoreModels, ShEx is one coder among fourteen format keys arranged around a single neutral representation: JSON Schema, Avro, JSON-LD, SQL DDL in three dialects, OWL in Turtle, LinkML, Protocol Buffers, the Bitol Open Data Contract Standard, the Apache Ossie semantic model, the Sage Bionetworks Synapse profile, and MACH Alliance ODM entity documents. Every format converts to and from the hub; none converts pairwise.

That architecture is the ecosystem position. Implementing ShEx did not add one integration; it added a spoke, and a spoke reaches every other spoke the day it lands. A consortium's shapes can become warehouse DDL; a relational model can become shapes for a submission review; a streaming record schema and a curator's shape file can be shown to be projections of one definition.

Directionality is part of the honesty. ShEx is a full round trip — import and export both first class. Some spokes are not: ODM entity documents decode only, being authored documentation rather than a generated artifact; the Synapse profile encodes only, its output being plain draft-07 JSON Schema that returns through the JSON Schema key. Both are labeled rather than discovered at deployment.

The same governed graph is where the operational estate meets the semantic one: artifacts that warehouse, streaming, orchestration, and clinical-data tooling already produce can be audited against it, read-only, for coverage, drift, and conformance — so a model defined in shapes can be checked against the tables that actually exist.

Why neutrality is structural, not a slogan

Every hub that translates between formats is one incentive away from becoming a silo. Privilege your own dialect, make exports subtly lossy in ways you decline to itemize, and people who arrived for interoperability stay because leaving got expensive. Our posture is the opposite, built into the mechanics rather than asserted in marketing.

We implement published specifications as written. We do not govern ShEx, LinkML, the Open Data Contract Standard, Apache Ossie, or the ODM — we implement them and track what their published artifacts say. The ShExC we emit is ordinary standard text: prefix declarations, shapes, triple constraints, cardinality markers. There is no proprietary wrapper dialect, and the coder knows nothing about mapping languages, because translation and reinterpretation are separate concerns meeting only at the neutral model.

Both directions stay first class, so the exit door is always open: a model that entered as shapes can leave as shapes, carrying the prefixes, predicates, and datatype tokens it arrived with. Where a translation genuinely loses something — a record format that cannot express a graph constraint, a relational target that cannot enforce an allowed-value list — the loss is itemized instead of smoothed away. An implementation with an agenda hides its losses; a neutral one publishes them.

Neutrality matters most to those with the least leverage: the engineer who inherited a shape file and does not speak RDF, and the curator asked to feed a warehouse they do not control. Both need a translator with no stake in which side wins. The linked-data bet — that IRIs are the sturdiest way to carry meaning across a boundary — is a good one, and the record-shaped world's bets on enforcement and throughput are good too. A schema hub earns its keep by honoring both and profiting from neither's captivity.

The full format wheel, ShEx included, has quickstarts at https://coremodels.example.com.