dbt to CoreModels: Zero to First Audit

Your dbt project already produces everything CoreModels needs. There is no agent to install, no warehouse credential to hand over, and no dbt Cloud connection to configure — the integration works entirely from build artifacts that dbt writes to your target/ directory on every run. In this tutorial we take a real dbt project from nothing to a completed schema audit: extract the artifacts, import the estate into a governed CoreModels project, run the first audit, and read the result line by line.

The vendor key is dbt, and the connector supports all three capabilities: Import, Audit, and Generate.

What you need

• A dbt project that compiles (dbt Core; manifest schema v10–v12, which covers dbt 1.6 through current releases).
• A CoreModels project to govern the estate. You need its 32-character hex project id — we use $PROJECT_ID below.
• A CoreModels login token ($TOKEN). Import requires the Admin role on the project; audit only needs Viewer.
• curl and jq.

Your API base URL is written as https://coremodels.example.com throughout — substitute your deployment's host.

Step 1 — Extract the artifacts

Nothing here talks to CoreModels yet. You are just asking dbt for files it produces anyway:

dbt parse                # or compile/run/build — writes target/manifest.json
dbt docs generate        # optional — writes target/catalog.json

Artifact	File	Why
manifest	target/manifest.json	Required. Any dbt command produces it; dbt parse is the cheapest.
catalog	target/catalog.json	Optional. Adds warehouse-real column types on top of what the manifest declares.
semantic_manifest	target/semantic_manifest.json	Optional. Produced when you define semantic models.

No credentials are ever shared with CoreModels — you upload build artifacts dbt already produces. That is a design decision, not a limitation: the connector never connects to a warehouse and never holds a secret.

Step 2 — Import the estate

The import call sends the artifact contents as strings inside a JSON body. A manifest is itself JSON, so it needs to be escaped as a string value — jq --rawfile does that correctly and never passes the file through the shell's argument list:

jq -n --rawfile manifest target/manifest.json \
      --rawfile catalog  target/catalog.json \
  '{artifacts: {manifest: $manifest, catalog: $catalog}}' > import-request.json

curl -sS -X POST \
  "https://coremodels.example.com/graph/integrations/dbt/import/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  --data-binary @import-request.json

If you skipped dbt docs generate, drop the catalog line — the manifest alone is enough.

A successful import answers with counts plus two honesty channels:

{ "success": true, "vendor": "dbt", "projectName": "jaffle_shop",
  "datasetsAdded": 9, "datasetsSkippedExisting": 0, "fieldsAdded": 0,
  "lineageEdgesAdded": 11, "lineageEdgesSkipped": 0, "nodesEnriched": 51,
  "snapshotStored": true, "lossiness": [], "errors": [] }

How to read it:

• datasetsAdded — models, seeds, snapshots, and sources that became governed Types, arriving complete with their columns as Elements. accepted_values tests became Taxonomies, relationships tests became references, and parent_map became lineage edges. (fieldsAdded counts only columns added to already-governed datasets on a re-import, so it reads 0 on a first import.)
• datasetsSkippedExisting — import is additive. Anything already governed is left exactly as it is; re-importing never mutates or deletes existing governed nodes. Changes surface through the audit instead — applying a meaning change is a human act.
• snapshotStored: true — the parsed estate snapshot was persisted, which is what enables one-call re-audits later without fresh artifacts.
• lossiness — anything the import approximated or dropped, stated explicitly. A successful import can still be honest about approximation. errors means could-not-proceed.

Step 3 — Run the first audit

The audit compares fresh artifacts against the governed model. It is strictly read-only and runs at Viewer role. Setting recordHistory: true additionally appends the run to the project's rolling audit trail — that is opt-in bookkeeping, never governed meaning:

jq -n --rawfile manifest target/manifest.json \
  '{artifacts: {manifest: $manifest}, recordHistory: true}' > audit-request.json

curl -sS -X POST \
  "https://coremodels.example.com/graph/integrations/dbt/audit/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  --data-binary @audit-request.json > audit-response.json

jq '{errorCount: .errorCount, warningCount: .warningCount, infoCount: .infoCount, codes: .codes}' audit-response.json

Immediately after an import of the same artifacts, expect a quiet report — the governed model and the estate agree, because one was just built from the other. The interesting runs come later, when either side moves.

Step 4 — Read the result

The response carries machine-readable counts, individual findings, and a markdown field holding a human-readable report ready to paste into a pull request. Every finding lives in one of three sections:

• Coverage — what exists in the artifacts but is not governed yet: dataset-unmapped, field-unmapped, projection-unmapped. New models you have not imported show up here, not as errors.
• Drift — where the estate and the governed model disagree: dataset-removed, contract-drift, field-removed, field-type-drift, enum-constraint-removed, enum-narrowed, enum-widened.
• Conformance — dbt best-practice rules the connector contributes: contract-not-enforced, contract-column-missing-type, key-column-untested, source-no-freshness.

Severities are Error, Warning, and Info. The single number that matters for automation is errorCount: greater than zero means the artifacts violate governed meaning. A real drift finding looks like this:

{ "section": "Drift", "severity": "Error", "code": "field-type-drift",
  "subject": "model.jaffle_shop.orders.order_total",
  "message": "Field type changed since the last import.",
  "detail": "governed: NUMBER(38,2), estate: VARCHAR" }

The subject is the dbt-native identity — model.jaffle_shop.orders.order_total — so the engineer who owns the model knows exactly where to look. The detail states both sides of the disagreement: the governed model recorded this column as NUMBER(38,2) at the last import; the manifest now declares VARCHAR. Someone changed the column type without changing the agreed meaning, and the audit caught it.

Two more useful fields: codes aggregates finding counts per code (handy for dashboards), and fingerprint is a content hash of the artifacts, so you can tell whether two audit runs actually looked at the same estate.

What you have now, and where this goes

At this point your dbt estate is governed and you have a repeatable, credential-free check. Three one-liners extend it:

# Re-audit the stored snapshot against the CURRENT governed model — no artifacts needed
curl -sS -X POST "https://coremodels.example.com/graph/integrations/dbt/reaudit/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{}'

# The rolling audit trail your recordHistory runs feed
curl -sS "https://coremodels.example.com/graph/integrations/dbt/history/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN"

# An SVG status badge from the latest recorded run
curl -sS "https://coremodels.example.com/graph/integrations/dbt/badge/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN"

The audit you just ran is also the exact payload a CI gate uses: a machine-to-machine surface at POST /v1/{projectId}/integrations/dbt/audit accepts user API keys and returns the same report, so every pull request can be checked before merge. And because the connector also supports Generate, the loop closes in the other direction: CoreModels can emit a schema.yml with enforced model contracts back into your dbt project, built from the governed model itself.

One honest caveat for very large estates: when the encoded snapshot exceeds the storage cap (about 1.5 MB), the import reports snapshotStored: false with a lossiness record. Fresh-artifact audits keep working exactly as above; only the artifact-free reaudit has nothing stored to run against.

For the full route list, CI recipes, and the generate call, see the self-contained dbt quickstart that ships with CoreModels.