One character, one broken study: automating cBioPortal drift checks in CI

Here is a pull request diff from a study repository:

-#STRING	NUMBER	STRING	STRING
+#STRING	STRING	STRING	STRING

One token changed on one header row. This is version two of a published study — the routine update every active cohort ships — and nothing in the pipeline will object. A standard review scans past it. validateData.py, rightly the format authority, checks that v2 is a well-formed study, not that v2 still means what v1 meant. Incremental loading moves the files without asking either. No tool diffs a study update against the published version's semantics — so the attribute that used to be numeric is now text, and every consumer that treated it as a number — a cohort filter, a survival plot, an export into an analysis pipeline — is wrong in a way that surfaces weeks later as a bug report about a chart.

A drift gate answers that diff in ninety seconds, before the importer runs. Retype a governed attribute and the audit returns field-type-drift, naming both types; drop or rename one and it returns field-removed — the semantic diff of v2 against governed meaning, delivered on the pull request. The same gate catches what the portal's validator would only reject later: an attribute ID that is not UPPER_CASE comes back as attribute-id-not-upper, and the workflow's fail-on setting decides whether warnings like it merely annotate the run or fail it outright. This article builds that gate for a cBioPortal study repository, then adds the three pieces that turn a one-shot check into a system: a badge, a rolling history, and re-audit — including on a schedule.

The route CI calls

Automation uses the machine-to-machine surface, which accepts revocable user API keys instead of interactive login tokens:

POST https://coremodels.example.com/v1/{projectId}/integrations/cbioportal/audit

It is Viewer role, because the audit never writes model shape. The response is wrapped in the standard API envelope, so every field of the report sits under data:

{
  "success": true,
  "data": {
    "vendor": "cbioportal",
    "projectName": "example_brca_2026",
    "errorCount": 1,
    "warningCount": 1,
    "infoCount": 1,
    "codes": { "field-type-drift": 1, "attribute-id-not-upper": 1, "attribute-no-description": 1 },
    "driftedObjects": ["patient.AGE"],
    "fingerprint": "c4a71f2e9b380d56",
    "findings": [
      { "section": "Drift", "severity": "Error", "code": "field-type-drift", "subject": "patient.AGE", "message": "Field type changed since the last import.", "detail": "governed: NUMBER, estate: STRING" },
      { "section": "Conformance", "severity": "Info", "code": "attribute-no-description", "subject": "patient", "message": "1 clinical attribute(s) carry no description row — curators downstream will guess.", "detail": "OS_STATUS" },
      { "section": "Conformance", "severity": "Warning", "code": "attribute-id-not-upper", "subject": "sample.cancerType", "message": "Attribute IDs must be UPPER_CASE for cBioPortal validation to pass.", "detail": null }
    ],
    "markdown": "…",
    "historyRecorded": true,
    "lossiness": []
  }
}

The gate is one comparison: data.errorCount > 0 means this pull request violates governed meaning. Fail the build.

The workflow

Three settings do all the configuration: a repository secret COREMODELS_API_KEY holding a user API key with Viewer access to the governing project, and two repository variables, COREMODELS_API_URL and COREMODELS_PROJECT_ID.

name: CoreModels schema audit
on:
  pull_request:
    paths:
      - "**/data_clinical_*.txt"
      - "**/meta_study.txt"

permissions:
  contents: read
  pull-requests: write

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Audit the study schema against CoreModels
        env:
          COREMODELS_API_URL: ${{ vars.COREMODELS_API_URL }}
          COREMODELS_API_KEY: ${{ secrets.COREMODELS_API_KEY }}
          COREMODELS_PROJECT_ID: ${{ vars.COREMODELS_PROJECT_ID }}
          FAIL_ON: error          # set to 'warning' to gate harder
        run: |
          set -euo pipefail

          # Header block only — data rows are ignored by the connector anyway.
          head -5 data_clinical_patient.txt > patient.head
          head -5 data_clinical_sample.txt  > sample.head

          jq -n \
            --rawfile patient patient.head \
            --rawfile sample  sample.head \
            --rawfile meta    meta_study.txt \
            '{artifacts: {clinical_patient: $patient, clinical_sample: $sample, meta: $meta},
              recordHistory: true}' > audit-request.json

          status=$(curl -sS -o audit-response.json -w "%{http_code}" -X POST \
            "$COREMODELS_API_URL/v1/$COREMODELS_PROJECT_ID/integrations/cbioportal/audit" \
            -H "Authorization: Bearer $COREMODELS_API_KEY" \
            -H "Content-Type: application/json" \
            --data-binary @audit-request.json)

          if [ "$status" != "200" ]; then
            echo "::error::CoreModels audit call failed with HTTP $status"
            head -c 2000 audit-response.json || true
            exit 1
          fi

          if [ "$(jq -r '.success' audit-response.json)" != "true" ]; then
            echo "::error::CoreModels audit returned an error: $(jq -r '.error.message // "unknown"' audit-response.json)"
            exit 1
          fi

          errors=$(jq -r '.data.errorCount' audit-response.json)
          warnings=$(jq -r '.data.warningCount' audit-response.json)
          drifted=$(jq -r '(.data.driftedObjects // []) | join(",")' audit-response.json)

          jq -r '.data.markdown' audit-response.json >> "$GITHUB_STEP_SUMMARY"

          # One annotation per finding, on the right severity channel.
          jq -r '.data.findings[] | "\(.severity)|\(.code)|\(.subject)|\(.message)"' audit-response.json |
          while IFS='|' read -r severity code subject message; do
            case "$severity" in
              Error)   echo "::error title=$code::$subject — $message" ;;
              Warning) echo "::warning title=$code::$subject — $message" ;;
            esac
          done

          if [ -n "$drifted" ]; then echo "Drifted objects: $drifted"; fi

          fail=0
          if [ "$errors" -gt 0 ]; then fail=1; fi
          if [ "$FAIL_ON" = "warning" ] && [ "$warnings" -gt 0 ]; then fail=1; fi
          if [ "$fail" = "1" ]; then
            echo "::error::This change violates governed meaning in CoreModels. See the audit report."
            exit 1
          fi

Points worth calling out. The paths filter keeps the job off pull requests that do not touch the schema. head -5 sends only the header block, so patient data never leaves the runner even though the audit is schema-only anyway. jq --rawfile handles the tab and newline escaping and keeps the file contents out of argv. The markdown report goes to the job summary, and the per-finding annotations attach the exact code and subject to the run.

The CoreModels repository also ships a composite CoreModels Schema Audit action, but its inputs (manifest-path, catalog-path) build a dbt-shaped request body. For a cBioPortal study the direct step above is the right shape — it posts to the same v1 route with the clinical artifact names.

What errorCount actually counts for this connector

Findings sort into three sections — Coverage, Drift, Conformance — at three severities, and only Error increments errorCount. For cBioPortal in practice:

Code	Section	Severity	Fires when
field-type-drift	Drift	Error	a datatype row changed for a governed attribute
field-removed	Drift	Error	a governed attribute is gone from a submitted instrument's file
dataset-unmapped	Coverage	Warning	an instrument in the files is not governed yet
field-unmapped	Coverage	Info	an attribute was added since the last import
attribute-id-not-upper	Conformance	Warning	an attribute ID is not UPPER_CASE
attribute-no-description	Conformance	Info	attributes on an instrument have no description row

So a default gate fails only on genuine meaning changes: retyped, removed, or vanished. The two cBioPortal conformance rules are advisory by design — attribute-id-not-upper is the one that will break portal validation later, which is why teams commonly flip FAIL_ON to warning once the existing attribute IDs are clean.

Three absences are worth knowing. contract-drift cannot fire here, because staging files carry no contract-enforcement flag. The enum-* codes only apply where a governed attribute has been given a controlled value list by hand — clinical staging files declare no accepted values, so a hand-added taxonomy will report enum-constraint-removed (Warning) on every audit until someone reconciles the two. And the shared dataset-removed code, which fires for warehouse estates when a whole governed table vanishes, does not apply to this connector: removal is scoped by identity namespace, and a cBioPortal instrument identity (patient, sample) is its own namespace, so an instrument you did not submit is simply out of scope for the comparison.

That scoping is a feature for CI. Auditing a partial artifact set is safe: send only clinical_patient on a pull request that only touches the patient file, and the governed sample instrument is left alone rather than reported as missing.

The badge

Every recorded run feeds an SVG status badge, available on both surfaces:

curl -sS "https://coremodels.example.com/v1/$PROJECT_ID/integrations/cbioportal/badge" \
  -H "Authorization: Bearer $COREMODELS_API_KEY" -o schema-audit.svg

It is labeled cbioportal audit and color-coded from the latest recorded run: green when clean, yellow when the run had warnings only, red with the error count when it had errors, gray when nothing has been recorded yet. The route is authenticated like every other route, so a bare image tag in a public README will get a 401 — fetch the SVG in the workflow and publish it where your readers are, or render it behind whatever proxy already fronts your internal docs.

The badge is only as current as your history, which is why recordHistory: true in the gate matters: it is the call that gives the badge something to show.

The rolling history

curl -sS "https://coremodels.example.com/graph/integrations/cbioportal/history/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" | jq '.projects[].runs'

[
  {
    "at": "2026-08-03T11:02:47.1180930+00:00",
    "trigger": "ci",
    "errorCount": 1,
    "warningCount": 1,
    "infoCount": 1,
    "codes": { "field-type-drift": 1, "attribute-id-not-upper": 1, "attribute-no-description": 1 },
    "fingerprint": "c4a71f2e9b380d56"
  },
  {
    "at": "2026-08-02T22:15:03.4470010+00:00",
    "trigger": "scheduled",
    "errorCount": 0,
    "warningCount": 1,
    "infoCount": 1,
    "codes": { "attribute-id-not-upper": 1, "attribute-no-description": 1 },
    "fingerprint": "5b19ee74c0a3f2d8"
  }
]

Runs are newest first, one trail per vendor-side study, capped at the 50 most recent. Each record is deliberately compact — timestamp, trigger, counts, codes, fingerprint — never the full findings. The trail exists to answer "is this study drifting over time?", not "what exactly is wrong right now", which is what a live audit is for. Triggers are audit (interactive), ci (the API-key route), reaudit, and scheduled.

The fingerprint is a content hash of the audited artifacts, so identical fingerprints across runs mean the files did not change — useful when a run's counts move but the files did not, which is exactly the signal that the model changed.

Re-audit: drift from the other side

The gate answers whether fresh files still conform to the governed model. The opposite question — does the governed model still match the last-known study? — matters when the model changes rather than the files. Re-audit runs the same engine over the snapshot stored at import time against the current governed view. No artifacts, no credentials:

curl -sS -X POST \
  "https://coremodels.example.com/graph/integrations/cbioportal/reaudit/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{}' | jq '{errorCount, warningCount, codes}'

It always records its run, so it moves the badge and shows in the history like any other. It lives on the interactive surface only. Its one prerequisite is a stored snapshot: if the import reported snapshotStored: false — which happens when a very large study pushes the encoded snapshot past the storage cap — fresh-artifact audits still work perfectly, but there is nothing for re-audit to run against, and the call says so.

A natural place to call it is the workflow that merges governance changes, or a nightly job in the repository that owns the CoreModels project.

The scheduled heartbeat

Re-audit can also run server-side without any repository at all. CoreModels includes a config-gated recurring worker that re-audits opted-in projects against their stored snapshots and appends each run to the same history, with the trigger scheduled. It is off by default and enabled per project by a CoreModels operator, because it writes history entries.

Turned on, it changes the failure mode of the whole system. Without it, model-side drift stays invisible until the next pull request touches the study — which for a completed cohort might be never. With it, the badge goes yellow or red on its own cadence, and the history shows the date the governed model and the last-known study parted ways. The CI gate catches file-side drift, the heartbeat catches model-side drift, the history records both, and the badge is the one-glance summary of where things stand.

For the extraction recipe behind the request bodies above, see the cBioPortal quickstart in the CoreModels integration docs.