dbt logoUse case

Your Contracts Stop at sources.yml. Your Drift Doesn't.

Every dbt engineer has lived this incident. An upstream team renames a column — or the ingestion tool retypes one, or a platform migration drops one — and the first anyone hears of it is a failed `dbt run` at 6 a.m.: `invalid identifier 'ORDER_TOTAL'` somewhere in a staging model. The upstream change may have been perfectly legitimate. The problem is where it was discovered: at runtime, in your project, by the on-call.

Your Contracts Stop at sources.yml. Your Drift Doesn't.

Every dbt engineer has lived this incident. An upstream team renames a column — or the ingestion tool retypes one, or a platform migration drops one — and the first anyone hears of it is a failed dbt run at 6 a.m.: invalid identifier 'ORDER_TOTAL' somewhere in a staging model. The upstream change may have been perfectly legitimate. The problem is where it was discovered: at runtime, in your project, by the on-call.

Now inventory the defenses dbt offers, and notice that every one of them is on the wrong side of the boundary. Model contracts are explicitly not applicable to sources — they protect what dbt builds, not what dbt reads. Source freshness checks whether data arrived, not whether its shape changed. Schema tests fire during a run, which is to say: after the drift has already landed. Observability tools do honest work here — they will tell you a column was deleted or retyped against a learned baseline — but detection at runtime is still an autopsy. The community's remaining move is defensive SQL: wildcard-free staging models, try_cast everywhere, macros that tolerate missing columns. That is coping, dressed as engineering.

The structural gap is that sources.yml is a claim about someone else's schema, and nothing binds the claim to the schema. The warehouse DDL is the truth; the sources file is a hopeful copy; and the two drift apart in the dark between dbt runs.

Govern both sides of the boundary

The CoreModels answer is to make the upstream schema a governed estate in its own right, sitting next to the dbt estate in the same workspace:

  1. Import the dbt estate from target/manifest.json — models, sources, columns, lineage, the standard flow.
  2. Import the upstream truth. For a warehouse that is a couple of SELECTs against INFORMATION_SCHEMA (the Snowflake, BigQuery, and Fabric/SQL Server connectors each document theirs) — saved query output, no credentials handed to CoreModels. For streaming sources, the Confluent connector imports schema-registry subjects the same artifact-first way.
  3. Reconcile. One call links each dbt source to the physical relation it reads, with reciprocal same-as assertions on both nodes. The pinning is explicit and reviewable — and the unmatched remainder on both sides is part of the report, so a source that reads a table nobody governs is itself a finding, not a silent gap.
  4. Gate on the upstream estate's own cadence. This is the part that changes the failure mode. The warehouse estate re-audits on a schedule (or from CI, or whenever fresh DDL output is posted) — independently of anyone running dbt. When the upstream schema moves, the audit raises drift findings with stable codes naming the exact table and column, the badge goes red, and the opt-in Slack notification fires on the clean-to-dirty transition. The rename that used to be a 6 a.m. runtime failure is now a governance finding raised by the next scheduled audit — attributable, dated, and visible before the next dbt run walks into it.

The asymmetry this removes is worth stating plainly: dbt teams currently get fast feedback on their own changes (CI on every PR) and no feedback on upstream changes until impact. Governing both estates gives the upstream boundary the same machinery the project side already enjoys — audits, history, badge, alerts — without asking the upstream team to adopt anything, because the artifacts come from the warehouse's own information schema.

What this is not

Honesty about scope, since the failure mode here is overpromising. This is not a live tap on the warehouse — imports are artifact-first, so the drift is caught on your audit cadence (scheduled heartbeat, CI, or manual), not the millisecond DDL executes. It does not block the upstream team's change — it has no write access to anything, by design. And it does not repair your staging models — it tells you precisely what moved, in which table, before the runtime failure finds you. Prevention here means moving discovery from run time to review time; the fix remains an engineer's decision.

One boundary, two estates, one referee: the dbt project says what it expects, the warehouse says what it is, and the governed model holds both to their word — with the disagreement surfacing as findings instead of incidents.

The per-connector import calls and the reconcile verb are documented in the vendor quickstarts that ship with CoreModels; the cross-estate walkthrough (dbt beside a warehouse, reconciled and audited) is the worked example.