Governing Airflow with an AI Agent: The CoreModels MCP Tools in Practice

"Which of our pipelines have no accountable owner, and is anything downstream of a paused DAG?" That is a governance question, and until recently answering it meant a human clicking through the Airflow UI and cross-referencing a wiki. With CoreModels, an AI agent answers it directly: the same import/audit machinery we expose over HTTP is exposed over the Model Context Protocol, so an agent connected to your CoreModels server can inspect connector capabilities, run a full drift-and-hygiene audit of an Airflow deployment, and read back a human-quality report — all through typed tool calls.

This article shows the exact tools, their exact arguments, and the flow for large payloads. Everything here is the Apache Airflow connector (vendor key airflow, capabilities Import and Audit — no Generate, and the tools tell the agent so rather than improvising).

Connecting an agent

CoreModels serves MCP over stateless streamable HTTP with OAuth 2.0. There are two endpoints with a deliberate split:

• /mcp — the public endpoint; read-only tools only (everything that runs at Viewer role).
• /mcp-admin — additionally serves the write tools; per-project role checks still apply on every call.

For Claude Code:

claude mcp add --transport http coremodels https://coremodels.example.com/mcp

then complete the OAuth flow (/mcp inside Claude Code). For claude.ai or Claude Desktop, add a custom connector pointing at the same URL. Generic MCP clients configure:

{
  "mcpServers": {
    "coremodels": { "type": "http", "url": "https://coremodels.example.com/mcp" }
  }
}

Dynamic Client Registration is supported, so no pre-registered client id is needed; unauthenticated requests get a 401 with discovery metadata and a spec-compliant client handles the rest.

The key security property for the rest of this article: the audit tool is read-only and runs at Viewer role, so it is available on the public endpoint. The import tool requires Admin membership on the target project and is served on /mcp-admin only. An agent can observe governance freely; changing the governed graph is a deliberately narrower door.

Discovery: what can this server govern?

The agent's first call is usually get_vendor_integration_status. Without a vendor argument it lists every registered connector with capabilities and expected artifacts:

{ "graphProjectId": "3f2a9c81b4de4f6aa1c2d3e4f5a6b7c8" }

The Airflow entry in the result carries "capabilities": "Import, Audit" and the artifact notes — dags required, tasks and datasets optional — so the agent learns the artifact contract without any documentation lookup. With a vendor, the same tool returns the project's last-import state:

{ "graphProjectId": "3f2a9c81b4de4f6aa1c2d3e4f5a6b7c8", "vendor": "airflow" }

{ "vendor": "airflow", "imported": true, "state": { "…": "…" }, "governedDatasets": 14 }

A result without imported: true tells the agent an import has to happen first; governedDatasets tells it how much of the estate resolves to governed Types.

The audit: audit_vendor_project

This is the workhorse — the free Schema Audit as a tool call. Required arguments are graphProjectId (the 32-character hex project id) and vendor; artifacts come inline via artifacts or by reference via artifactUrls:

{
  "graphProjectId": "3f2a9c81b4de4f6aa1c2d3e4f5a6b7c8",
  "vendor": "airflow",
  "artifacts": {
    "dags": "<the GET /api/v1/dags response>",
    "datasets": "<the GET /api/v1/datasets response>"
  }
}

The result gives the agent both machine and human channels: errorCount, warningCount, infoCount, metrics, a findings array (section, severity, code, subject, message, detail), and markdown — the complete report ready to paste into a PR or a chat reply. For Airflow, the findings the agent should expect are the connector's four conformance rules — dag-no-description (Info), dag-no-owner (Warning), asset-unproduced (Warning), paused-producer (Warning) — plus the shared coverage and drift codes once the estate and the governed model diverge. errorCount > 0 is the same CI-gate fail signal the HTTP surface uses, so an agent can act as a release gatekeeper with one comparison.

So the opening question — unowned pipelines, staleness downstream of paused DAGs — is answered by one tool call: dag-no-owner findings name the unaccountable DAGs, and paused-producer findings name each asset whose every producer is paused while active DAGs still consume it, with the producers and active consumers listed in the finding's detail.

Large deployments: the artifactUrls flow

A dags response for a big deployment can be far larger than an agent wants to inline in a tool call. For that, every artifact-bearing integration tool accepts artifactUrls — artifact name to URL — and the server fetches the content itself:

{
  "graphProjectId": "3f2a9c81b4de4f6aa1c2d3e4f5a6b7c8",
  "vendor": "airflow",
  "artifactUrls": {
    "dags": "https://artifacts.example.com/exports/airflow/dags.json",
    "tasks": "https://artifacts.example.com/exports/airflow/tasks.json",
    "datasets": "https://artifacts.example.com/exports/airflow/datasets.json"
  }
}

You can mix the two: small artifacts inline, the big one by URL. These are server-side fetches of caller-supplied URLs, so they are SSRF-guarded, and the guard is worth knowing because refusals are visible to the agent:

• https only — any other scheme is refused;
• redirects are disabled (a redirect to an internal host would bypass any pre-flight check);
• hosts resolving to loopback, link-local (cloud metadata), or private ranges are refused;
• responses are size-capped (256 MB) with a 60-second timeout.

Fetch failures do not silently vanish: each one lands in a fetchProblems array in the tool result (for example, Refused to fetch 'dags' from http://…: only https URLs are allowed), so the agent can tell the difference between "your estate is clean" and "I never saw your estate."

Importing over MCP: import_vendor_project

When status comes back without imported: true, the governed model has to be built first. The import tool takes the same arguments as the audit tool — graphProjectId, vendor, artifacts and/or artifactUrls, optional spaces — but it is an Admin-role tool served only on /mcp-admin:

{
  "graphProjectId": "3f2a9c81b4de4f6aa1c2d3e4f5a6b7c8",
  "vendor": "airflow",
  "artifacts": { "dags": "<dags.json>", "tasks": "<tasks.json>", "datasets": "<datasets.json>" }
}

The result reports datasetsAdded, datasetsSkippedExisting, fieldsAdded, lineageEdgesAdded, lineageEdgesSkipped, nodesEnriched, plus lossiness, errors, and fetchProblems. Import is additive on re-import — already-governed nodes are never mutated — which makes it safe for an agent to re-run: the way to see what changed is audit_vendor_project, not a destructive re-import. That division of labor is stated in the tools' own descriptions, so a well-behaved agent discovers it from tools/list alone.

One thing an agent cannot do for Airflow is generate artifacts. The connector's capabilities are Import and Audit, and the capability gate refuses before any emission code runs: generate_vendor_artifacts returns an explicit error — "capabilities: Connector 'airflow' does not support generation." The agent already learned the why from discovery, where the Airflow entry declares exactly two capabilities. We consider that refusal a feature: an agent told plainly that something is unsupported reasons correctly instead of retrying.

A governance loop an agent can run

Putting it together, a practical agent playbook for an Airflow estate:

1. get_vendor_integration_status with vendor: "airflow" — is the estate imported, and how much is governed?
2. If not imported (and the agent has admin access): import_vendor_project with the three artifacts.
3. audit_vendor_project on fresh artifacts — read errorCount for the gate, findings for specifics.
4. Summarize from markdown, name owners from dag-no-owner subjects, flag every paused-producer asset with its producers and consumers from detail.

Steps 1, 3, and 4 run against the public read-only endpoint with Viewer membership — which means you can hand a governance agent to anyone on the team without handing them write access to the governed model.

The artifact extraction recipe (three curl calls against your own Airflow REST API — CoreModels never holds your Airflow credentials) is in the Apache Airflow quickstart in the CoreModels integration docs.