Following one clinical attribute all the way into the graph

Take a single column out of a cBioPortal patient staging file:

#Age
#Age at diagnosis in years
#NUMBER
#1
AGE

Five cells. By the time CoreModels has finished importing the study, those five cells have become a governed Element with a floating-point value type, a vendor identity that survives renaming, a metadata record holding the literal string NUMBER and the display name Age, an entry in a persisted estate snapshot, and a line item in every future drift audit. This article traces that path for every construct the connector recognizes — what the parser extracts, where each fact lands, which audit codes can fire against it, and exactly where meaning is approximated or dropped.

The parse contract

The connector reads a clinical file as: every leading line that starts with # is a header row; the first non-blank line after them is the attribute-ID row; everything below is data and is discarded. Fewer than four header rows, or no attribute row, and the file is rejected outright with Not a clinical staging file (needs 4 '#' header rows above the attribute-ID row).

Column by column, the mapping is positional:

Header row	Becomes
0 — display names	cbio.displayName in the field's meta bag
1 — descriptions	the field's description
2 — datatypes	the field's native type, upper-cased; empty cell defaults to STRING
3 — priorities	cbio.priority in the field's meta bag
attribute-ID row	the field's name — the identity everything else keys off

Rows are read defensively. A header row shorter than the attribute row yields empty cells for the columns it does not reach — which is why an attribute with no description is a normal parse outcome that raises a conformance finding rather than an error. Blank attribute IDs are skipped; duplicates are skipped case-insensitively, first occurrence winning. A file yielding no usable attributes is treated as not-a-staging-file. One trap: every leading # line counts as a header row, so a comment line pasted above the block shifts the positional mapping.

Each file becomes one dataset in the neutral estate model: vendor id patient or sample, kind Table, materialization clinical, description cBioPortal clinical {instrument} attributes. The payload is fingerprinted with a truncated SHA-256, so two submissions compare cheaply.

Two structural facts are recognized during the parse. PATIENT_ID in the patient file and SAMPLE_ID in the sample file each gain two normalized checks, Unique and NotNull, both named identity. And the sample file's PATIENT_ID is rewritten into a reference: it keeps its description and datatype, gains a Relationship check, and points at the patient instrument's PATIENT_ID. That rewrite only happens when a patient dataset is present in the same submission — audit a sample file on its own and the column stays a plain string.

The optional meta file contributes three keys and nothing else: cancer_study_identifier becomes the study name (defaulting to cbioportal-study when absent), name becomes a studyName fact, type_of_cancer becomes a cancerType fact. Two more facts are derived: instruments and attributes.

Identity: what the graph keys on

Node ids in CoreModels must be camelCase alphanumerics, so vendor identities are sanitised on the way in and preserved elsewhere. For our study:

Vendor identity	Graph node id	Node kind
patient	patient	Type
sample	sample	Type
patient.AGE	patientAGE	Element
patient.PATIENT_ID	patientPATIENTID	Element
sample.cancerType	sampleCancerType	Element

The id is a convenience. The identity that matters is the mapsTo assertion written on every node — a two-property mixin value whose standard is the vendor key and whose uri is the vendor identity:

{ "transformMapsToStandard": "cbioportal", "transformMapsToUri": "patient.AGE" }

That is the same carrier every other CoreModels import writes, so a vendor identity survives a format export intact. Audits match on it, never on labels — so an SME can relabel AGE to Age at diagnosis and the next audit still recognizes the same governed attribute. It also means a genuine rename in the staging file is real drift, which is correct: renaming an attribute ID breaks every consumer keyed on it.

One consequence deserves a warning label. The dataset identity is the bare instrument name, patient or sample, not something study-scoped — so two studies imported into one project claim the same identities. Govern one study per project. Sanitisation is also collision-checked: two distinct vendor identities that would collapse onto one node id make the import refuse and name both, rather than silently merging two attributes.

Two write paths

Schema shape — Types, Elements, value types, references — flows through a snapshot-to-IR mapper and out through the shared CoreModels writer, the same path every other format import takes. Estate facts the IR deliberately cannot hold are written natively afterwards.

Estate fact	Where it lands
clinical file	Type node, label = instrument name
clinical attribute	Element on that Type
NotNull from an identity check	a Required element fact, stashed on the shared element-facts mixin
sample's PATIENT_ID	expected-type relation to the patient Type
vendor identity	mapsTo mixin value {cbioportal, <vendor id>}
native type, description, display name, priority, checks, materialization	the cBioPortal Metadata mixin
study id/name/cancer type, counts, fingerprint, timestamp	the integration state node
parsed estate	the estate snapshot node (gzip+base64, size-capped)
recorded audit runs	the audit history node (rolling, 50 runs)

The metadata mixin is where the vendor's own vocabulary survives. Its property ids are the vendor key plus the property name, so the value written for patient.AGE reads:

{
  "cbioportalDataType": "NUMBER",
  "cbioportalDescription": "Age at diagnosis in years",
  "cbioportalTags": "",
  "cbioportalMeta": "{\"cbio.displayName\":\"Age\",\"cbio.priority\":\"1\"}",
  "cbioportalChecks": ""
}

And for patient.PATIENT_ID, whose identity checks are normalized:

{
  "cbioportalDataType": "STRING",
  "cbioportalDescription": "Unique patient identifier",
  "cbioportalTags": "",
  "cbioportalMeta": "{\"cbio.displayName\":\"Patient Identifier\",\"cbio.priority\":\"1\"}",
  "cbioportalChecks": "[{\"kind\":\"Unique\",\"name\":\"identity\",\"severity\":null,\"detail\":null},{\"kind\":\"NotNull\",\"name\":\"identity\",\"severity\":null,\"detail\":null}]"
}

The instrument Type carries the dataset-level twin: cbioportalUniqueId (patient), cbioportalResourceKind (table), cbioportalMaterialization (clinical), cbioportalContract (false), and empty strings for the properties cBioPortal has no concept of — cbioportalRelationName, cbioportalAccess. Every property is rewritten on every import, with "" for absent, so a fact that disappears from the estate cannot linger as stale metadata.

That refresh is the one thing an import updates in place, and it is deliberate: mixin values mirror the estate, they are not governed meaning. Governed nodes themselves are never mutated or deleted by an import. Detecting a meaning change is the audit's job; applying it is a human's.

The identity checks deserve one more sentence, because they land in two places. PATIENT_ID's NotNull sets the element's Required fact, which the schema writer persists on a shared element-facts mixin — visible in the product like any other mixin value, but a stash rather than a first-class metamodel slot, so the graph does not enforce it. The authoritative, machine-readable copy of the check is the cbioportalChecks JSON above — the vendor-side record every import refreshes.

Type mapping in both directions

cBioPortal datatype	Governed value type	Approximated
NUMBER	Double	yes — one type covers integers and decimals alike
BOOLEAN	Boolean	no
STRING, or an empty cell	String	no
anything else	String	yes

Both approximations produce a lossiness record naming the path and the original string, and the literal native type is preserved on the metadata mixin so audits compare NUMBER against NUMBER rather than Double against NUMBER.

Generation inverts the map: Integer or Double becomes NUMBER, Boolean becomes BOOLEAN, everything else becomes STRING. The inversion is not symmetric, and that is the honest headline about round-tripping. A generated scaffold takes its display-name row from the governed Element label, not from the cbio.displayName the import captured; it emits priority 1 for every column; and it has no slot for identity checks, because a staging header block has nowhere to put them. Descriptions do survive, read back from the metadata mixin, falling back to the label when absent.

What the audit can and cannot say

The shared engine contributes coverage and drift codes; the connector contributes conformance rules. For a cBioPortal estate the live set is:

• Coverage — dataset-unmapped (Warning): an instrument in the files is not governed. field-unmapped (Info): an attribute exists in the files but not on the governed Type.
• Drift — field-removed (Error): a governed attribute is gone from a submitted instrument. field-type-drift (Error): the datatype row moved away from what the last import recorded, with the comparison run on the raw strings (governed: NUMBER, estate: STRING), not on the approximated value types.
• Conformance — attribute-no-description (Info), raised once per instrument with the offending attribute names in the detail; attribute-id-not-upper (Warning), raised per attribute whose ID contains anything other than upper-case letters, digits, and underscores.

Four shared codes cannot meaningfully fire here, and knowing why is knowing the engine. projection-unmapped needs semantic models or exposures, which staging files do not have. contract-drift needs a contract-enforcement flag, which staging files do not have; the connector always records false, so governed and estate always agree.

dataset-removed is the interesting one. The engine scopes removal by identity namespace — everything before the last dot — so a project governing several estates of one vendor never reports another estate's tables as deleted. A cBioPortal instrument identity, patient or sample, has no dot in it, so each identity is its own namespace: an instrument you did not submit is out of scope rather than removed, and the code never fires here. The payoff is that auditing only clinical_patient is safe, which is what you want on a pull request that touched one file.

The enum-* codes need accepted values, and clinical staging files declare none — with one important exception. If a human governs a clinical attribute with a taxonomy (a controlled list of OS_STATUS values, say), every subsequent audit will report enum-constraint-removed (Warning), because the estate side genuinely declares no value set. That is the engine telling the truth about an asymmetry, not a false positive, but it is worth knowing before someone adds the taxonomy.

Limits, and where they bite

• Clinical files only. Mutation, CNA, and expression files are data, not schema, and are out of scope by design.
• No live portal connection. The connector declares Import, Audit, and Generate — not LiveSync. CoreModels never holds portal credentials; every verb runs on artifacts you supply.
• No lineage. A study carries no build DAG, so lineageEdgesAdded is always zero. The patient/sample join is a reference, not a lineage edge.
• The snapshot cap. The parsed estate is stored gzipped and base64-encoded for re-audit, capped at roughly 1.5 million encoded characters. Past that, the import reports snapshotStored: false with a lossiness record: fresh-artifact audits still work, but re-audit has no baseline.
• Additive re-import is conservative. When a later import adds an attribute to an instrument that is already governed, the attribute is created — but reference semantics are not auto-created for it. The import says so with a ConstraintRelaxation record explaining that the facts are kept on the metadata mixin and surfaced by the audit, because changing governed meaning is a human act.

None of these are hidden: each surfaces as a lossiness record on the call that hit it, or as a finding on the next audit. The connector reads exactly what a study's header block declares, flags every approximation, and puts each fact where it belongs — which is what gives a drift audit something precise to measure against.

For the artifact contract and the extraction recipe, see the cBioPortal quickstart in the CoreModels integration docs.