The Curation Checklist Does Not Scale

Ask how a cancer study gets into the public cBioPortal datahub and stays good, and the answer is a checklist. A GitHub issue opens, a curator works through the required files and formats, a reviewer checks boxes, a pull request merges. It is careful, human, and it works — once, per study, at submission time. Now ask the harder question. Fifteen published studies, three curators, standards that have moved twice since the oldest study went up: who re-checks study number seven?

Nobody does, is the honest answer, until something forces it. Conformance in this ecosystem is checked at the door and assumed thereafter. When already-published studies are brought up to current standards, it happens as a one-off campaign — the 375-study audit that found twenty-four encodings of radiation therapy was exactly such a campaign, a heroic manual pass that produced a paper, not a process. The supporting curation toolkit describes itself as a collection of ad-hoc scripts. The one place this problem is genuinely solved is GENIE, where Sage operates a central processing pipeline as institutional infrastructure — and that solution is available to exactly one consortium.

A conformance loop instead of a checklist

The CoreModels approach is to make each study's conformance a recorded, re-checkable state instead of a box ticked at submission.

Each study is its own governed estate. Import a study's clinical staging headers once and it gains what the checklist never records: an audit history of dated runs with finding counts and codes, a status badge that renders the latest recorded run — green, yellow, red, or gray — and a snapshot of the parsed study stored at import time. The snapshot is the piece that changes the economics. The study as last known now lives inside the system, so checking it again requires no files, no re-upload, and no curator's afternoon.

The standard moves; the sweep notices. Definitions get tightened, datatypes corrected, attributes deprecated — and no study file changes when that happens, so nothing repository-side will ever fire. A scheduled re-audit worker — config-gated, off by default, enabled per project by an operator — replays each opted-in study's stored snapshot against the current governed dictionary on a cadence and appends every run to the history. Study number seven gets re-checked because the sweep does not know it is anyone's least favorite study.

Transitions, not noise. An opt-in Slack notification fires when a study crosses from clean to dirty — the moment worth interrupting someone for — rather than on every run. The badge carries the current state in each study's README, and the history answers, with timestamps and finding codes, exactly when a study and the standard parted ways and which attributes are involved. "Are we still conformant?" stops being a question that launches a campaign and becomes a question the record already answers.

What this is not

It is not a GENIE replacement, and it is not trying to be. Sage's central pipeline is institutionalized, funded, and does far more than schema conformance — it processes the data itself, on a release cycle, for a consortium built around it. This loop is for everyone outside that arrangement: the groups whose alternative is not a central pipeline but a checklist and the hope that nothing has moved.

It is also artifact-first, which sets the honest boundary on freshness. The scheduled sweep replays each study as it stood at its last import, so it catches the dictionary moving away from the studies. A study whose own files changed is caught from the other side — when fresh headers are posted to the audit, from CI or by hand. The two cadences cover the two directions of drift between them, and neither is a live tap on anything: drift is found on the audit cadence, not the moment a file changes. And the sweep raises findings; it fixes nothing. Which study moves, and how, remains a curator's decision — made this month with a report in hand, instead of in three years with a spreadsheet and a grant deadline.

The import, re-audit, history, and badge mechanics are the standard connector flow, covered end to end in the cBioPortal quickstart that ships with CoreModels.