Automating Synapse Schema Releases: Determinism, Plan Replay, and Version Discipline

Run the same Synapse export twice against the same model and you get the same bytes. That single property — deterministic encoding — is what turns the CoreModels synapse profile from a converter into release machinery: outputs you can commit and diff, plans you can store and replay, and a version rule you can enforce mechanically instead of remembering. This article builds that pipeline piece by piece, with the honest constraints stated where they bite.

Standing constraint, restated because automation amplifies mistakes: synapse is encode-only. Pipelines emit Synapse schemas; anything coming back from Synapse re-enters through jsonschema. And two facts drive the whole design: Synapse rejects re-registering an existing version, and the registered $id's version segment must be plain major.minor.patch — a 1.0.0-rc1 cannot be expressed there, and the export will fall back to the default version and record the fallback rather than fail.

Why the output is diffable

The encoder's output is stable by construction. Keyword order is fixed — $schema and $id first, matching the shape of Synapse's publicly registered example schemas — the $defs-to-definitions rewrite is rule-based, and every stripped keyword produces exactly one ledger record with a deterministic path. The mapping engine underneath makes the same commitment: the same plan executed against the same source produces the same output, a property we prove in our test suite rather than assert in marketing.

The consequence: the exported schema is a git artifact. Commit it. A re-export that produces a zero diff means nothing changed and nothing needs registering; a non-empty diff is your signal to bump synapseVersion — because the platform will refuse the old one anyway, and a version bump without a diff creates two registered versions of the same thing.

The release gate, as a script

Everything below is one curl and a handful of jq assertions. Prerequisites: $TOKEN (Viewer on the project is enough — the export reads, never writes), $PROJECT_ID, and a $VERSION your release process owns.

#!/usr/bin/env bash
set -euo pipefail

HOST="https://coremodels.example.com"

RESPONSE=$(curl -s "$HOST/graph/transform/schema/export/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"format\":\"synapse\",
       \"synapseOrg\":\"myorg.dcc\",
       \"synapseName\":\"experimentalData.biopsy\",
       \"synapseVersion\":\"$VERSION\"}")

# 1. It ran.
test "$(jq -r '.success' <<<"$RESPONSE")" = "true"

# 2. The version we asked for actually reached the $id.
#    A non-semver VERSION does not fail the call - it falls back to the default
#    and records a SemanticNarrowing at #/$id. Make that a build failure here.
jq -e '[.lossiness[] | select(.path == "#/$id")] | length == 0' <<<"$RESPONSE" > /dev/null

# 3. No silently relaxed constraints. Drops and approximations are review items;
#    a ConstraintRelaxation means the registered schema enforces less than the model says.
jq -e '[.lossiness[] | select(.kind == "ConstraintRelaxation")] | length == 0' <<<"$RESPONSE" > /dev/null

# 4. Persist the artifact and the ledger next to each other.
jq '.schema'    <<<"$RESPONSE" > biopsy.synapse.json
jq '.lossiness' <<<"$RESPONSE" > biopsy.synapse.ledger.json

# 5. Deterministic output => a clean diff is "nothing to register".
if git diff --quiet -- biopsy.synapse.json; then
  echo "No schema change - do not bump the version, do not register."
else
  echo "Schema changed - register $VERSION (Synapse refuses to re-register an existing one)."
fi

Do not sort the JSON when persisting it (jq -S would): the encoder's key order is part of the determinism, identity-first like the platform's own fixtures, and sorting throws away a diff-stability property you are getting for free.

Gate 3 deserves a policy decision rather than a copy-paste. Failing on every ConstraintRelaxation is the strict posture: it means "our governed model contains rules the Synapse subset cannot enforce, and a human must acknowledge each one." Teams that have already acknowledged them can whitelist known paths instead — the paths are stable strings like #/properties/vialCount/multipleOf, which makes an allowlist trivial to maintain.

Plans: the replayable half of the machinery

When the pipeline maps between schemas — migrating a partner's format onto your model before the Synapse encode — the engine's plan is the artifact that makes the run reproducible. Every mapping response carries it:

{
  "operations": [
    {
      "kind": "TypeMapping",
      "origin": "Inferred",
      "sourceTypeId": "Biopsy",
      "targetTypeId": "Biopsy"
    },
    {
      "kind": "ElementMapping",
      "origin": "Inferred",
      "sourceElementIds": ["BiopsySampleId"],
      "targetElementIds": ["BiopsySampleId"]
    }
  ]
}

Operations are discriminated by kind — TypeMapping, ElementMapping, TaxonomyMapping, RelationMapping, ComponentMapping, Drop — and each is stamped with the origin of the strategy that produced it. A plan can also carry per-operation declaredLossiness, so a stored plan replays with the same honesty it was produced with.

Store the plan next to the schema artifact, and replay it with POST graph/transform/plan/execute/{projectId} — the plan travels as a JSON string inside the body:

{
  "sourceFormat": "sql",
  "sourceSchema": "CREATE TABLE Biopsy (\n  sample_id VARCHAR(64) NOT NULL,\n  stage VARCHAR(32)\n);",
  "plan": "{\"operations\":[{\"kind\":\"TypeMapping\",\"origin\":\"Inferred\",\"sourceTypeId\":\"Biopsy\",\"targetTypeId\":\"Biopsy\"},{\"kind\":\"ElementMapping\",\"origin\":\"Inferred\",\"sourceElementIds\":[\"BiopsySampleId\"],\"targetElementIds\":[\"BiopsySampleId\"]}]}",
  "targetFormat": "synapse"
}

A replayed plan gets no trust for having run before: it passes the same universal validation gate as a fresh one, then executes deterministically. That is the audit story in one sentence — the plan in your repository is the transformation, byte for byte, whenever it is re-run against the same source. One honesty note for this route: the stateless mapping endpoints encode synapse with the default $id segments; the release-grade $id comes from the project export above or the transform_schema MCP tool.

Choosing a mapping kind for unattended runs

Three strategies produce plans, and they automate differently:

• inferred — label/type matching against a target hint (caseInsensitive defaults to true). Zero configuration, ideal for pure format conversion where the source is its own hint. Its behavior changes only when the schemas change — acceptable in CI precisely because the plan comes back for inspection.
• explicit — you commit the mapping guide itself, the strongest reproducibility posture. The guide is SIA mapping-guide JSON: autoMatchByMapsTo, fieldMappings with optional transforms, taxonomyDirectives, drops — and unknown keys are rejected with a path-carrying error, because a silently ignored typo would execute a plan you did not intend:

{
  "autoMatchByMapsTo": true,
  "fieldMappings": [
    {
      "sourceElementIds": ["BiopsySampleId"],
      "targetElementIds": ["specimenId"]
    }
  ],
  "taxonomyDirectives": [
    {
      "sourceTaxonomyId": "BiopsyStage",
      "targetTaxonomyId": "stage",
      "treatment": "InlineEnum"
    }
  ],
  "drops": ["legacyField"]
}

• ai — a server-side model proposes the plan. For unattended pipelines, understand exactly what this kind is: it requires Editor/Admin project membership and a server-configured Anthropic key, it sends the schemas to the external Anthropic API server-side, its proposal passes the identical gate with at most one repair attempt, and any self-reported confidence on operations is advisory — the gate never trusts it. Our recommendation for scheduled jobs is to use ai interactively once, review the returned plan, commit it, and replay it as an explicit-style stored artifact forever after. The model proposes; the repository remembers.

The companion artifacts belong in the same run

The curation manifests — the blank entry grid and the column dictionary with its value-set section — are generated from the same source by the generate_synapse_manifests MCP tool, deterministically: column order follows the model (inherited elements first, parents leading), required flags and value sets come from the same facts the schema encode used. Regenerating them in the same pipeline run that produces biopsy.synapse.json is what guarantees the schema a validator enforces and the sheet a curator fills in can never describe two different models. Manifest drift is not detected by this pipeline; it is made impossible by it.

The discipline, summarized

Commit the schema, the ledger, and the plan. Diff before you bump; bump before you register. Fail the build when the $id is not the one you asked for, and decide deliberately which lossiness kinds a human must sign off. None of this requires trusting CoreModels (by ARAMAI) to be perfect — it requires the transform to be deterministic and honest about itself, which is exactly the contract the engine makes.