Tool Calls, Not Screenshots: Running an Airbyte Audit from an AI Agent

There are two ways to let an AI assistant help with an ingestion catalog. The first is to paste the catalog into a prompt and ask what looks risky — fast, ungrounded, unverifiable. The second is to give the assistant a tool that runs the real audit against your governed model and returns coded findings. This article is about the second one.

CoreModels exposes its vendor-integration verbs as Model Context Protocol tools. An agent connected over MCP can list what is registered, audit a fresh Airbyte catalog against the governed graph, import an estate, and reconcile it with another vendor — under the same role checks the HTTP surface enforces, with the read-only verbs staying read-only.

Connecting

The MCP server is served over stateless streamable HTTP at /mcp — read-only tools — and /mcp-admin, which additionally serves the write tools. Both are OAuth 2.0 protected resources.

# Claude Code, read-only:
claude mcp add --transport http coremodels https://coremodels.example.com/mcp

# Claude Code, with write access:
claude mcp add --transport http coremodels-admin https://coremodels.example.com/mcp-admin

Any client that takes JSON configuration wants the same two facts:

{
  "mcpServers": {
    "coremodels": { "type": "http", "url": "https://coremodels.example.com/mcp" }
  }
}

An unauthenticated request gets a 401 with a WWW-Authenticate header pointing at the protected-resource metadata (RFC 9728) published at /.well-known/oauth-protected-resource/mcp. A spec-compliant client takes it from there: discovery, Dynamic Client Registration — no pre-registered client id needed — and the PKCE (S256) authorization-code flow. Requested scopes are openid, profile, email, offline_access, mcp.

Be clear about one thing: the endpoint split is convenience, not the security boundary. The enforced boundary is the per-project role check. Write tools require Editor or Admin membership on the project regardless of which endpoint minted the token, and the public endpoint does not even list write tools in tools/list.

Every tool carries a title and an explicit readOnlyHint, plus destructive, idempotent and open-world hints where they apply, so an agent can reason about consequences before calling. audit_vendor_project is annotated open-world because it may fetch caller-supplied URLs; import_vendor_project declares destructiveHint: false because it never mutates governed nodes.

Step 1 — What is registered, and what has been imported

{
  "name": "get_vendor_integration_status",
  "arguments": { "graphProjectId": "b0f4e1c27a9d4f6e8b1c3d5a7e9f0a2b" }
}

Called with only a project id, the tool answers the discovery question — every registered connector with its capabilities and expected artifacts. Add vendor and it answers the state question instead:

{
  "name": "get_vendor_integration_status",
  "arguments": {
    "graphProjectId": "b0f4e1c27a9d4f6e8b1c3d5a7e9f0a2b",
    "vendor": "airbyte"
  }
}

{
  "vendor": "airbyte",
  "imported": true,
  "state": {
    "vendor": "airbyte",
    "projectName": "airbyte-connection",
    "importedAt": "2026-08-04T09:12:44.1183920+00:00",
    "sourceFingerprint": "9f2c41ab7d0e5b83",
    "counts": "fieldsAdded=0, lineageAdded=0, lineageSkipped=0, nodesEnriched=14",
    "facts": "{\"streams\":\"3\",\"fields\":\"11\"}"
  },
  "governedDatasets": 3
}

This is the call worth making first in any agent workflow. It tells the model whether a governed baseline exists at all — which decides whether the next step is an audit or an import — and it costs nothing, because it only reads state.

Step 2 — The audit

{
  "name": "audit_vendor_project",
  "arguments": {
    "graphProjectId": "b0f4e1c27a9d4f6e8b1c3d5a7e9f0a2b",
    "vendor": "airbyte",
    "artifacts": {
      "catalog": "{\"streams\":[{\"stream\":{\"name\":\"users\",\"namespace\":\"public\"}}]}"
    }
  }
}

The input schema requires graphProjectId (pattern ^[a-f0-9]{32}$) and vendor, then accepts artifacts, artifactUrls and spaces. It sets additionalProperties: false, so an invented argument name fails validation instead of being silently ignored — a small thing that matters when a model is composing the call itself.

The result:

{
  "vendor": "airbyte",
  "projectName": "airbyte-connection",
  "warningCount": 1,
  "infoCount": 2,
  "metrics": {
    "Datasets (estate)": "3",
    "Datasets governed": "3 / 3",
    "Fields governed": "11 / 11",
    "Governed nodes with canonical mappings": "0 / 15 (0%)",
    "Last import": "2026-08-04T09:12:44.1183920+00:00"
  },
  "findings": [
    {
      "section": "Conformance",
      "severity": "Warning",
      "code": "stream-no-primary-key",
      "subject": "events",
      "message": "Stream declares no primary key — dedup-dependent sync modes and downstream identity break silently."
    },
    {
      "section": "Conformance",
      "severity": "Info",
      "code": "untyped-fields",
      "subject": "events",
      "message": "1 field(s) are untyped or semi-structured — their inner schema enters the warehouse ungoverned.",
      "detail": "payload"
    }
  ],
  "markdown": "# airbyte Schema Audit — airbyte-connection\n\n🟡 **0 errors · 1 warnings · 2 info**\n…",
  "fetchProblems": []
}

errorCount carries the same meaning here as everywhere else: greater than zero means the catalog violates governed meaning. One serialization habit to know: MCP results omit zero counts and null fields (which is why errorCount and the first finding's detail are absent above) — treat a missing count as 0. markdown is the human report, ready to drop into a pull request or a chat thread. The tool writes nothing — it runs at Viewer role, which is exactly why it is served on the public /mcp endpoint.

Note what the MCP result does not carry compared with the HTTP response: no codes histogram, no driftedObjects, no fingerprint, no lossiness. An agent that needs those should aggregate findings itself or call the HTTP audit route. What MCP adds instead is fetchProblems.

Step 3 — Catalogs too large to inline

A wide source can produce a catalog no chat client will happily carry inside a tool call. That is what artifactUrls is for: name the artifact, give a URL, and the server fetches it.

{
  "name": "audit_vendor_project",
  "arguments": {
    "graphProjectId": "b0f4e1c27a9d4f6e8b1c3d5a7e9f0a2b",
    "vendor": "airbyte",
    "artifactUrls": {
      "catalog": "https://artifacts.example.com/airbyte/catalog-2026-08-04.json"
    },
    "spaces": []
  }
}

Both argument objects may be supplied together — content is assembled artifact by artifact, so inline for small pieces and URLs for large ones is a valid mix. Airbyte has exactly one artifact name, so in practice you choose one.

Because this is a server-side GET to a caller-supplied URL, it is guarded:

• https only. Any other scheme is refused.
• Redirects disabled. A redirect into an internal host would defeat any pre-flight check, so redirects are not followed at all.
• No private destinations. The host is resolved, and refused if any address is loopback, link-local (including the cloud-metadata range), or in a private range.
• Bounded. A 60-second timeout and a hard response-size cap.

Failures do not fail the whole call. They land in fetchProblems as readable strings the agent can relay verbatim:

{
  "fetchProblems": [
    "Refused to fetch 'catalog' from https://10.4.1.9/catalog.json: host resolves to a non-public address (10.4.1.9)"
  ]
}

If nothing usable was assembled at all, the tool returns an error naming both argument shapes rather than pretending an empty audit succeeded.

Step 4 — Importing (admin endpoint only)

{
  "name": "import_vendor_project",
  "arguments": {
    "graphProjectId": "b0f4e1c27a9d4f6e8b1c3d5a7e9f0a2b",
    "vendor": "airbyte",
    "artifactUrls": { "catalog": "https://artifacts.example.com/airbyte/catalog-2026-08-04.json" }
  }
}

Same argument shape, different posture: import_vendor_project requires the Admin role and is served on /mcp-admin only. It returns the import counters plus both diagnostic channels:

{
  "success": true,
  "vendor": "airbyte",
  "projectName": "airbyte-connection",
  "datasetsAdded": 3,
  "nodesEnriched": 14,
  "lossiness": [
    {
      "kind": "TypeApproximation",
      "path": "public.users.email",
      "explanation": "Native type 'email' was approximated as String; the exact native type is preserved in the vendor metadata mixin."
    }
  ],
  "errors": [],
  "fetchProblems": []
}

(The zero-valued counters — datasetsSkippedExisting, fieldsAdded, lineageEdgesAdded, lineageEdgesSkipped — are omitted from the serialized result, like every default value.) Re-import is additive: already-governed streams are skipped rather than overwritten. An agent that wants to know what changed should audit, not re-import — precisely the division of labour the tool descriptions state.

A second write tool matters when Airbyte is not your only estate:

{
  "name": "reconcile_vendor_projects",
  "arguments": {
    "graphProjectId": "b0f4e1c27a9d4f6e8b1c3d5a7e9f0a2b",
    "vendorA": "airbyte",
    "vendorB": "snowflake"
  }
}

It links datasets two estates govern as the same physical relation with reciprocal sameAs mappings, and it is idempotent — re-running refreshes the links instead of duplicating them.

What the agent cannot do

Being explicit about the edges of a tool surface is part of making it usable.

Generate is refused. generate_vendor_artifacts exists and works for other connectors; called with "vendor": "airbyte" it returns an error: capabilities: Connector 'airbyte' does not support generation. An Airbyte catalog describes what a source exposes, so authoring one from the governed model is not ours to do.

Re-audit, history and badge are not MCP tools. They live on the HTTP surface. An agent that needs the drift trail calls GET graph/integrations/airbyte/history/{projectId}; one that needs the artifact-free re-check calls the reaudit route.

No live Airbyte connection. CoreModels never holds your Airbyte credentials. Every tool above works from an artifact you produced, inline or by URL.

Where it gets interesting

Once a catalog is imported, the rest of the read-only tool surface applies to it, because the estate is now ordinary governed meaning rather than vendor-shaped trivia. get_project_summary lists the types, elements and taxonomies now in the project. get_mixins_and_relation_groups reveals the Airbyte metadata mixin and the lineage relation group. search_nodes finds a stream or a property by name. The export tools render the same governed model as JSON Schema, ShEx, Avro, LinkML, SQL, OWL or JSON-LD, and validate_json checks a document against the project's stored schema.

That is the payoff of routing an agent through tools instead of prompts. "Which streams have no primary key?" is answered by a coded finding from a real audit run. "What does status mean, and which values are allowed?" is answered from a governed taxonomy. Neither answer is a guess, and both are reproducible by anyone who runs the same call.

For the extraction recipe that produces the catalog these tools consume, see the Airbyte quickstart in the CoreModels integration docs.