ShEx in Ten Lines: Your First CoreModels Transform

The shortest useful thing you can do with CoreModels and ShEx takes one HTTP call, writes nothing to your project, and hands back three things: the converted schema, a replayable plan, and an honest ledger of everything the conversion could not carry across. This walkthrough runs that call twice — once into JSON Schema, once into Postgres DDL — and shows you how to read all three parts of the answer.

CoreModels (by ARAMAI) speaks ShEx compact syntax, ShExC, under the format key shex, and it speaks it in both directions: shex appears in both the decode list and the encode list of our transform engine, so shapes can be a source, a target, or both.

What you need

• Your CoreModels host. We use https://coremodels.example.com as the placeholder.
• A bearer token in $TOKEN. The call below is stateless and only needs Viewer on the project it is scoped to.
• A project id in $PROJECT_ID — a 32-character hex id. The stateless endpoints use it for authorization scoping only; nothing is read from or written to the graph.
• curl and jq.

The schema

Save this as library.shex. It is deliberately small but it exercises most of what a real shapes file does: a required literal, two optionals, a date, a value constraint that is not an xsd: datatype, a reference to another shape, and a repeated property.

PREFIX schema: <https://schema.org/>
PREFIX ex: <https://example.org/library#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>

ex:AuthorShape {
  schema:name xsd:string ;
  schema:email xsd:string ? ;
  schema:url IRI ? ;
  schema:birthDate xsd:date ? ;
  ex:wrote @ex:BookShape * ;
}

ex:BookShape {
  schema:isbn xsd:string ;
  schema:datePublished xsd:date ? ;
}

The call

POST /graph/transform/schema/map/{projectId} is the stateless engine endpoint: it decodes the source, produces a mapping plan, runs the plan through the validation gate, executes it, and encodes the result into the target format. For a straight format hop, the source document is also the target hint — you are mapping the model onto itself and changing only the wire format.

Build the body with jq so the ShExC text is escaped correctly:

jq -n --rawfile shex library.shex '{
  sourceFormat: "shex",
  sourceSchema: $shex,
  targetFormat: "jsonschema",
  targetHintFormat: "shex",
  targetHintSchema: $shex,
  mapping: { kind: "inferred", caseInsensitive: true }
}' > map-body.json

curl -s -X POST "https://coremodels.example.com/graph/transform/schema/map/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d @map-body.json

Headers: bearer token, JSON content type. That is the whole setup.

The output

The response envelope is the same on every transform endpoint: success, lossiness, errors, the payload (here schema), and — on the mapping endpoints — plan. The schema field is a JSON object for JSON-shaped targets and a string for text targets. Here is what comes back for jsonschema, abridged only by removing repeated x-maps-to blocks that follow the same pattern:

{
  "type": "object",
  "properties": {
    "name": { "type": "string", "x-maps-to": { "schema": "https://schema.org/name" } },
    "email": { "type": "string", "x-maps-to": { "schema": "https://schema.org/email" } },
    "url": { "type": "string", "x-maps-to": { "schema": "https://schema.org/url" } },
    "birthDate": { "type": "string", "x-maps-to": { "schema": "https://schema.org/birthDate" } },
    "wrote": {
      "type": "array",
      "items": { "$ref": "#/$defs/ex:BookShape" },
      "x-maps-to": { "ex": "https://example.org/library#wrote" }
    }
  },
  "required": ["name"],
  "x-maps-to": { "ex": "https://example.org/library#Author" },
  "$defs": {
    "ex:BookShape": {
      "type": "object",
      "properties": {
        "isbn": { "type": "string", "x-maps-to": { "schema": "https://schema.org/isbn" } },
        "datePublished": { "type": "string", "x-maps-to": { "schema": "https://schema.org/datePublished" } }
      },
      "required": ["isbn"],
      "x-maps-to": { "ex": "https://example.org/library#Book" }
    }
  }
}

Four things happened there, and every one of them is a ShEx construct landing in its JSON Schema counterpart.

Cardinality became structure. A triple constraint with no marker means exactly one, so schema:name is in required. The ? markers made email, url, and birthDate optional. The * on ex:wrote produced a JSON Schema array — collection-ness and optionality are tracked separately, so * (optional collection) and + (required collection) come out differently.

The shape reference stayed a reference. @ex:BookShape did not flatten into a string; it became a $ref into $defs, with the referenced shape emitted as its own object schema.

Property names came from predicate local names. schema:datePublished becomes the property datePublished; the shape ex:AuthorShape becomes a type labeled Author, with the conventional Shape suffix stripped.

The meaning came along for free. This is the part that makes ShEx an unusually good import format. A ShEx predicate is already an IRI: schema:name is https://schema.org/name. Our decoder expands each predicate against the document's own prefix map and lifts it straight into the model's cross-standard mapping annotation, which every other format then carries in its own idiom — here as x-maps-to. The shape name gets the same treatment: ex:AuthorShape validates instances of ex:Author, so the type maps to https://example.org/library#Author. No annotation syntax, no sidecar mapping file, no manual step.

The ledger

Now the part most conversion tools skip. Alongside the schema you get:

{
  "kind": "TypeApproximation",
  "path": "Element[ex:AuthorShape|schema:url]",
  "explanation": "ShEx value constraint 'IRI' has no IR primitive; approximated as String."
}

That is the lossiness ledger, and this entry is exactly right: IRI is a ShEx node constraint, not an xsd: datatype, and our neutral model has no IRI primitive — so schema:url traveled as a string, and we said so. Every record has the same three fields, kind, path, and explanation, and there are four kinds across the whole engine: StructuralDrop (something had no home in the target and was left out), TypeApproximation (a close-but-not-exact target type was used), ConstraintRelaxation (a rule could not be enforced and was relaxed), and SemanticNarrowing (meaning was narrowed or guessed).

The habit to build on day one: success: true does not mean "nothing changed". It means "it ran". The ledger is the honest list of what changed, and an empty ledger is the only claim of a clean conversion. Note also that the ledger is cumulative — it aggregates records from decoding the source, producing the plan, the validation gate, execution, and encoding the target, so you see everything from one read.

Same source, narrower target

Change one field and the same source lands somewhere much narrower. Re-run with targetFormat set to sql, and add "vendor": "postgres" (the default; mysql and sqlserver are the alternatives):

jq -n --rawfile shex library.shex '{
  sourceFormat: "shex",
  sourceSchema: $shex,
  targetFormat: "sql",
  vendor: "postgres",
  targetHintFormat: "shex",
  targetHintSchema: $shex,
  mapping: { kind: "inferred" }
}' > sql-body.json

curl -s -X POST "https://coremodels.example.com/graph/transform/schema/map/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d @sql-body.json | jq -r '.schema'

The schema field is a string this time, and jq -r prints it as the DDL it is:

CREATE TABLE "Author" (
  "name" VARCHAR(255) NOT NULL,
  "email" VARCHAR(255),
  "url" VARCHAR(255),
  "birthDate" TIMESTAMP,
  "wrote" INTEGER REFERENCES "Book"
);

CREATE TABLE "Book" (
  "isbn" VARCHAR(255) NOT NULL,
  "datePublished" TIMESTAMP
);

COMMENT ON COLUMN Author.name IS '{"x-maps-to":{"schema":"https://schema.org/name"}}';

The required marker became NOT NULL, the shape reference became a foreign key, xsd:date became TIMESTAMP, and the schema.org meaning rode into the database as a column comment — so even in Postgres, Author.name still says out loud which concept it holds. (Comments continue for every mapped element; one is shown here.) The IRI record is still in the ledger, because it happened when the source was read, not when the target was written — and had this model contained a value set, Postgres would have added a ConstraintRelaxation of its own, since it has no inline enum to hold one.

Keeping it

Everything above is stateless. When you want the model governed rather than converted, post the same document to the import endpoint, which needs Admin on the project:

jq -n --rawfile shex library.shex '{ format: "shex", schema: $shex }' \
  | curl -s -X POST "https://coremodels.example.com/graph/transform/schema/import/$PROJECT_ID" \
      -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d @-

You get the same envelope with projectId in place of schema, and the shapes land as governed Types, Elements, and — where you used value sets — Taxonomies, with the predicate IRIs preserved as mappings. From there, exporting to any of our other formats is a single call with { "format": "..." }, and { "format": "shex" } regenerates ShExC from any project, whatever format its model was born in.

For the rest of the transform surface — target hints against other formats, authored mapping guides, and replaying a stored plan — see the transform quickstarts in the CoreModels docs.