The Dictionary Changed on a Tuesday: CI Drift Gates for REDCap

REDCap's greatest operational strength is also its governance problem: a data manager with the right permissions can change a production instrument in minutes. A validation type quietly switches from integer to plain text, a choice list gains a code, a required flag disappears — and every downstream extract, harmonization script, and statistical pipeline inherits the change without a review. The fix isn't to slow REDCap down. It's to make the data dictionary a versioned, gated artifact like any other piece of production configuration. This article builds that gate with CoreModels: a CI audit that fails on drift, a status badge, a rolling history, one-call re-audits, and the scheduled heartbeat that catches drift nobody pushed.

The setup: version the dictionary

Everything downstream assumes one habit: the current data_dictionary.csv lives in a Git repository, and changes to it arrive as pull requests. Producing that file is either the Project Setup → Data Dictionary download in the REDCap UI or a single content=metadata, format=csv POST to your REDCap API, run wherever your REDCap token already lives — the token is never sent to CoreModels, and the exact command is in our quickstart. Commit that file. From now on, "the schema changed" is a diff, and the gate below decides whether the diff violates governed meaning.

The gate: one job, three assertions

CI calls the machine-to-machine surface (/v1/...), which accepts user API keys — create one with Viewer access (the audit never writes, so Viewer genuinely suffices) and store it as a repository secret. The whole gate is curl and jq:

name: schema-audit
on:
  pull_request:
    paths: ["data_dictionary.csv"]

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: CoreModels Schema Audit
        env:
          COREMODELS_API_URL: ${{ vars.COREMODELS_API_URL }}
          COREMODELS_API_KEY: ${{ secrets.COREMODELS_API_KEY }}
          COREMODELS_PROJECT_ID: ${{ vars.COREMODELS_PROJECT_ID }}
        run: |
          set -euo pipefail
          jq -n --rawfile dd data_dictionary.csv \
            '{artifacts: {data_dictionary: $dd}, recordHistory: true}' > audit-request.json
          curl -sS -o audit-response.json -X POST \
            "$COREMODELS_API_URL/v1/$COREMODELS_PROJECT_ID/integrations/redcap/audit" \
            -H "Authorization: Bearer $COREMODELS_API_KEY" -H "Content-Type: application/json" \
            --data-binary @audit-request.json
          test "$(jq -r '.success' audit-response.json)" = "true"
          jq -r '.data.markdown' audit-response.json >> "$GITHUB_STEP_SUMMARY"
          test "$(jq -r '.data.errorCount' audit-response.json)" -eq 0

Three assertions, in order. First, success must be true — a false here means the audit could not run at all (unusable CSV, unknown vendor), which is itself a red flag worth failing on. Second, the markdown report is appended to the job summary, so a reviewer reading the failed check sees the human-readable findings without leaving GitHub. Third — the gate itself — data.errorCount must be zero.

A note on packaging, since we would rather be useful than tidy: we do ship a composite GitHub Action for this audit, but it is dbt-shaped — it insists on a compiled manifest.json path and assembles a dbt artifact body. For REDCap, the dozen lines above are the supported form. They call exactly the route the action calls, and they are trivial to port to GitLab CI, Azure Pipelines, or a cron box with curl and jq on it.

What errorCount actually means

The audit classifies findings into three severities, and the counters are the contract:

• Error — the dictionary violates governed meaning. Drift findings like field-type-drift (a variable's type no longer matches the governed element), field-removed, dataset-removed, or enum-narrowed (a governed value gone from a choice list) land here. errorCount > 0 ⇒ fail the build.
• Warning — suspicious but not meaning-breaking. The other enum drift lands here — enum-widened (new values in the estate that aren't governed terms) and enum-constraint-removed (the estate stopped declaring the value set) — and for REDCap the standing example is duplicate-choice-codes: a choice list reuses a code, so exported data for that field is ambiguous. Warnings don't fail the gate above; if you want a stricter posture, add test "$(jq -r '.data.warningCount' audit-response.json)" -eq 0.
• Info — inventory and hygiene. phi-fields (the per-instrument PHI inventory) and text-no-validation (free-text fields with no validation type) will appear on most real REDCap projects. They are context for reviewers, not gate conditions.

Note that on the v1 surface every field sits under data.* because responses are wrapped in the standard API envelope — hence jq -r '.data.errorCount', not .errorCount.

If you want a policy finer than a single number, the response carries the material for it. data.codes is a per-code tally, so a team can gate one rule specifically while it cleans up the rest:

test "$(jq -r '.data.codes["duplicate-choice-codes"] // 0' audit-response.json)" -eq 0

And data.driftedObjects lists the vendor identities behind the Drift findings — demographics.dob, visit — which makes a one-line failure message that names names:

jq -r '"Drifted: " + ((.data.driftedObjects // []) | join(", "))' audit-response.json

Because the request sets recordHistory: true, every gate run is also appended to the project's audit history with the trigger ci. That single flag turns the gate from a point-in-time check into a trail.

The badge

The latest recorded run — whatever its trigger — drives an SVG status badge on the same API-key surface:

curl -sS -H "Authorization: Bearer $COREMODELS_API_KEY" \
  "https://coremodels.example.com/v1/$COREMODELS_PROJECT_ID/integrations/redcap/badge" \
  -o badge.svg

Green means the last recorded audit was clean, yellow means warnings only, red means errors, gray means no recorded runs yet. It's a self-contained shields-style SVG, embeddable wherever you can attach the API key — a docs portal, an internal dashboard, or a badge-refresh step in the pipeline that commits the SVG next to the README.

The rolling history

The history endpoint (interactive surface, Viewer role) returns the recent runs, newest first, for each REDCap project governed in the CoreModels project:

curl -sS -H "Authorization: Bearer $TOKEN" \
  "https://coremodels.example.com/graph/integrations/redcap/history/$PROJECT_ID" \
  | jq '.projects[0].runs | map({at, trigger, errorCount, warningCount, fingerprint})'

Each run carries its trigger (ci from the gate, audit from interactive runs, reaudit from the verb below), the three severity counters, per-code finding counts, and the artifact fingerprint — a content hash of the dictionary that was audited. The fingerprint is quietly useful: two consecutive runs with the same fingerprint but different counts mean the governed model moved, not the dictionary.

Reaudit: the drift nobody pushed

The CI gate covers one direction of drift — a changed dictionary arriving as a PR. The other direction has no PR at all: someone edits the governed model in CoreModels, and the last-known estate silently stops conforming. That's what reaudit is for. It replays the audit engine over the dictionary snapshot stored at import time against the current governed model:

curl -sS -X POST \
  "https://coremodels.example.com/graph/integrations/redcap/reaudit/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{}'

No artifacts, no REDCap export, no credentials — one call, Viewer role. Reaudit always records its run in the history (trigger reaudit), so governed-side drift shows up in the same trail and the same badge as estate-side drift. A sensible habit: trigger a reaudit after any deliberate change to the governed model, and treat a red result as "the estate needs to catch up or the change needs a rethink."

One prerequisite to know about: reaudit depends on the snapshot stored at import. When a dictionary is very large (over roughly 1.5 MB encoded), the import reports snapshotStored: false with a lossiness record; fresh-artifact audits — including the whole CI gate above — keep working, but reaudit has nothing to run against and tells you so rather than guessing: No stored estate snapshot for vendor 'redcap' — import the vendor project first (imports persist the parsed snapshot). Note also that reaudit and history live on the interactive surface with a login token; the v1 API-key surface deliberately carries only audit and badge.

The heartbeat

The last piece is time itself. Estates drift while everyone is busy; a gate that only runs on PRs never sees it. CoreModels ships a scheduled, server-side re-audit — a config-gated background worker (Integrations:ScheduledReaudit, off by default) that periodically re-audits opted-in projects against their stored snapshots and records the runs into the same history with the trigger scheduled. No CI minutes, no exported artifacts, no credentials: the same reaudit verb, on a clock. With the heartbeat on, the badge becomes genuinely trustworthy as a passive indicator — gray-to-green-to-red transitions happen even when no human has thought about the schema in weeks.

The loop, assembled

Dictionary changes arrive as PRs and get gated on errorCount. Every gate run lands in the history as ci. Governed-model changes get a one-call reaudit, recorded as reaudit. The heartbeat re-checks on a schedule. And the badge summarizes the latest of all of it in one color. None of these steps writes to the governed model, none of them holds REDCap credentials, and every one of them is replayable from a CSV you already version-control.

The condensed recipe — including the exact CI step above — is in the REDCap quickstart in the CoreModels docs (docs/quickstarts/redcap).