The CoreModels BigQuery Integration, Route by Route

The complete HTTP surface for governing a BigQuery estate: every verb, every role, every payload.

The BigQuery connector in CoreModels (by ARAMAI) declares three capabilities — Import, Audit, and Generate — and the platform exposes them through two HTTP surfaces. The interactive surface under graph/integrations/... authenticates with your normal CoreModels login token and carries the working verb set: import, audit, reaudit, history, badge, generate, status, plus cross-vendor reconciliation and connector discovery (a sync-proposal surface also lives here, beyond this article's scope). The machine-to-machine surface under v1/... accepts user API keys and deliberately carries only what CI needs: audit and badge. This article walks the whole surface with real payloads. $TOKEN is your bearer credential; $PROJECT_ID is the 32-character hex id of the governing CoreModels project; the host is a placeholder.

One posture rule governs everything below: import writes to the graph, additively, and nothing else writes at all. Audit and generate are read-only; recording an audit run in the history is opt-in bookkeeping, and only the reaudit verb records its run unconditionally.

Discovery

GET https://coremodels.example.com/graph/integrations/vendors
Authorization: Bearer $TOKEN

Any authenticated user can call this. It lists every registered connector with its key, display name, capability flags, and expected artifacts. The BigQuery entry looks like this:

{
  "key": "bigquery",
  "displayName": "Google BigQuery",
  "capabilities": "Import, Audit, Generate",
  "artifacts": {
    "information_schema": "required — JSON rows of INFORMATION_SCHEMA COLUMNS × TABLES (+ field-path descriptions, table options); documented query"
  }
}

That artifacts map is the contract: one required artifact, information_schema, produced by the documented per-dataset extraction query (see the quickstart for the SQL). A fourth capability, LiveSync, exists in the model but is deliberately deferred — CoreModels never holds your Google credentials, so every verb below works on artifacts you extract yourself.

Import — POST graph/integrations/bigquery/import/{projectId} (Admin)

POST https://coremodels.example.com/graph/integrations/bigquery/import/$PROJECT_ID
Authorization: Bearer $TOKEN
Content-Type: application/json

{ "artifacts": { "information_schema": "<extract rows JSON as a string>" } }

The body is an ArtifactsRequest: artifacts maps artifact name to raw content; an optional spaces array targets specific space ids (empty means the project's main space). Import is additive — already-governed nodes are never mutated; drift is the audit's job. The response returns success, vendor, projectName, then counters (datasetsAdded, datasetsSkippedExisting, fieldsAdded, lineageEdgesAdded, lineageEdgesSkipped, nodesEnriched), a snapshotStored flag, and the two channels every verb shares: lossiness (what was approximated, on success) and errors (what prevented progress). For BigQuery, lineage counters are always zero — the information_schema extract carries no lineage.

Omitting the artifacts is a structured failure, not a 500:

{
  "success": false,
  "lossiness": [],
  "errors": [
    { "path": "artifacts",
      "message": "Body must include 'artifacts': { \"<name>\": \"<content>\" } (e.g. manifest for dbt)." }
  ]
}

An unknown vendor key fails the same way, with the message listing every registered key.

Audit — POST graph/integrations/bigquery/audit/{projectId} (Viewer)

Same body shape as import, plus one flag: recordHistory (default false) appends the run to the rolling audit trail with trigger audit. Without it, the audit verb stays strictly read-only. The response is the full audit report:

{
  "success": true,
  "vendor": "bigquery",
  "projectName": "my-gcp-project",
  "errorCount": 0,
  "warningCount": 1,
  "infoCount": 3,
  "codes": { "semi-structured-column": 1, "table-no-description": 3 },
  "driftedObjects": [],
  "fingerprint": "9c2f4e1a0b7d3c58",
  "metrics": {
    "Datasets (estate)": "12",
    "Datasets governed": "12 / 12",
    "Fields governed": "84 / 84",
    "Governed nodes with canonical mappings": "0 / 96 (0%)",
    "Last import": "2026-07-20T09:14:22.6182740+00:00"
  },
  "findings": [
    { "section": "Conformance", "severity": "Info", "code": "table-no-description",
      "subject": "my-gcp-project.analytics.raw_events",
      "message": "Table/view has no description — undocumented datasets resist governance and agent grounding.",
      "detail": null }
  ],
  "markdown": "<the PR-comment-ready report>",
  "historyRecorded": false,
  "lossiness": []
}

Findings are grouped in three sections — Coverage (codes dataset-unmapped, field-unmapped), Drift (dataset-removed, field-removed, field-type-drift, enum-constraint-removed, enum-narrowed, enum-widened, contract-drift), and Conformance, where BigQuery contributes table-no-description (Info) and semi-structured-column (Warning). errorCount > 0 is the CI-gate fail condition. metrics is a string map of the headline numbers — estate size, governed coverage, how many governed nodes carry a non-vendor canonical mapping, and when the estate was last imported. The fingerprint is a short content hash of the artifact, so two runs with the same fingerprint audited byte-identical extracts.

Reaudit — POST graph/integrations/bigquery/reaudit/{projectId} (Viewer)

The audit above asks "do these fresh artifacts still conform to the governed model?" Reaudit asks the mirror question: the governed model changed — does the last-known estate still conform? It runs the same audit engine over the estate snapshot stored at import time against the current governed model. No artifacts needed:

POST https://coremodels.example.com/graph/integrations/bigquery/reaudit/$PROJECT_ID
Authorization: Bearer $TOKEN
Content-Type: application/json

{}

The optional body fields are projectName (which stored snapshot to use; null means the latest), spaces, and notifySlack (post to the project's configured Slack webhook when the run has errors or warnings; default false). Unlike audit, reaudit always records its run in the history, with trigger reaudit. It fails honestly when no snapshot is available — for example when the import reported snapshotStored: false because the encoded snapshot exceeded the storage cap (~1.5 MB).

History — GET graph/integrations/bigquery/history/{projectId} (Viewer)

The rolling audit trail, newest first, one entry per recorded run:

{
  "success": true,
  "vendor": "bigquery",
  "projects": [
    {
      "projectName": "my-gcp-project",
      "runs": [
        { "at": "2026-07-29T06:41:00.4413094+00:00", "trigger": "reaudit", "errorCount": 1,
          "warningCount": 1, "infoCount": 3,
          "codes": { "field-type-drift": 1, "semi-structured-column": 1, "table-no-description": 3 },
          "fingerprint": "9c2f4e1a0b7d3c58" }
      ]
    }
  ]
}

The trigger values tell you how each run happened: audit (interactive, recordHistory: true), ci (the v1 surface), reaudit, or scheduled (the optional server-side re-audit heartbeat). Timestamps are UTC ISO-8601 and sort lexicographically.

Badge — GET graph/integrations/bigquery/badge/{projectId} (Viewer)

Returns image/svg+xml — a shields-style badge labeled bigquery audit, rendered from the latest recorded run: green clean, yellow warnings only, red errors, gray when nothing is recorded. Even an unknown vendor key returns a gray badge rather than a JSON error, so whatever consumes the badge always gets an image.

Generate — POST graph/integrations/bigquery/generate/{projectId} (Viewer)

BigQuery is a full-loop connector: the governed model generates DDL back out.

POST https://coremodels.example.com/graph/integrations/bigquery/generate/$PROJECT_ID
Authorization: Bearer $TOKEN
Content-Type: application/json

{ "typeNames": [] }

typeNames restricts generation to named types (empty means everything eligible); targetVersion and extra exist in the request shape for vendors that need dialect switches — BigQuery currently needs none. The response:

{
  "success": true,
  "artifacts": [
    { "name": "coremodels_bigquery_tables.sql", "kind": "sql",
      "content": "-- Generated by CoreModels — governed BigQuery table definitions.\n..." }
  ],
  "lossiness": [
    { "kind": "StructuralDrop", "path": "daily_summary",
      "explanation": "Views are derived objects; DDL generation covers tables only." }
  ],
  "errors": []
}

One artifact comes back: CREATE TABLE IF NOT EXISTS statements with NOT NULL from governed checks and OPTIONS(description=...) on both columns and tables. BigQuery enforces no primary keys, foreign keys, or CHECK constraints, so governed taxonomies and references are written into the descriptions — where BigQuery users actually read — never faked as constraints. Views are skipped with a declared lossiness record, as above. If nothing is eligible, generate fails with an explicit error rather than emitting an empty file.

Status — GET graph/integrations/bigquery/status/{projectId} (Viewer)

{ "success": true, "vendor": "bigquery", "imported": true,
  "state": { "...": "last-import bookkeeping: versions, timestamps, fingerprint, counts" },
  "governedDatasets": 10 }

imported tells you whether an import state exists at all; governedDatasets counts vendor-identified datasets currently resolvable as governed Types.

Reconcile — POST graph/integrations/reconcile/{projectId} (Admin)

The one route that is not vendor-scoped. If two connectors govern the same physical relation — say a BigQuery table and the transformation-tool model that materializes it — reconciliation links the pairs as one entity via reciprocal sameAs assertions. Body: { "vendorA": "bigquery", "vendorB": "<other key>" }. It is idempotent and reports the matched pairs, the fields they matched on, and what remained unmatched on each side.

The v1 machine surface

Two routes, both Viewer role, both working with user API keys — this is what CI and scripts should call:

• POST v1/{projectId}/integrations/bigquery/audit — same body as the interactive audit; the response is wrapped in the ApiResponse envelope, so every field above lives under data.* (data.errorCount > 0 means fail the build). With recordHistory: true the run lands in the history with trigger ci.
• GET v1/{projectId}/integrations/bigquery/badge — the same SVG badge for READMEs and dashboards.

Reaudit and history intentionally stay on the interactive surface; the v1 integration surface carries exactly the two verbs automation needs — the gate and its badge.

That is the working surface — the interactive verbs above, two machine routes, one connector key — everything the import–audit–generate loop needs. For the extraction recipe and a step-by-step first run, see the Google BigQuery quickstart in the CoreModels documentation.