API

The AWS Glue Integration API, Route by Route

CoreModels models vendor governance as a small, fixed set of verbs and applies them uniformly to every connector. For AWS Glue Data Catalog — vendor key `glue` — that is eight core routes on the interactive surface and two on the machine-to-machine surface. (A newer sync-plan surface — `sync/propose`, plan fetch, and the sync ledger — sits beside these and is not covered here.) This is the reference: the exact paths, the role each one enforces, the request bodies as the API defines them, and the responses you actually get back.

The AWS Glue Integration API, Route by Route

CoreModels models vendor governance as a small, fixed set of verbs and applies them uniformly to every connector. For AWS Glue Data Catalog — vendor key glue — that is eight core routes on the interactive surface and two on the machine-to-machine surface. (A newer sync-plan surface — sync/propose, plan fetch, and the sync ledger — sits beside these and is not covered here.) This is the reference: the exact paths, the role each one enforces, the request bodies as the API defines them, and the responses you actually get back.

Both surfaces authenticate with Authorization: Bearer …. The interactive surface (graph/integrations/…) expects your normal CoreModels login token; the v1 surface accepts user API keys, which is what makes it the right target for pipelines and dashboards. Throughout, https://coremodels.example.com stands in for your API base URL, $TOKEN for the token, and $PROJECT_ID for the 32-character hex project id.

Route	Role	Writes?
`GET graph/integrations/vendors`	any authenticated	no
`POST graph/integrations/glue/import/{projectId}`	Admin	yes — additive only
`POST graph/integrations/glue/audit/{projectId}`	Viewer	history, only if asked
`POST graph/integrations/glue/reaudit/{projectId}`	Viewer	history, always
`GET graph/integrations/glue/history/{projectId}`	Viewer	no
`GET graph/integrations/glue/badge/{projectId}`	Viewer	no
`POST graph/integrations/glue/generate/{projectId}`	Viewer	no
`GET graph/integrations/glue/status/{projectId}`	Viewer	no
`POST v1/{projectId}/integrations/glue/audit`	Viewer (API key)	history, only if asked
`GET v1/{projectId}/integrations/glue/badge`	Viewer (API key)	no

Three request bodies, reused everywhere

The artifact-bearing routes (import and audit, on both surfaces) share one shape:

{
  "artifacts": { "tables": "<raw contents of tables.json>" },
  "spaces": [],
  "recordHistory": false
}

artifacts maps artifact name to raw content; Glue defines a single name, tables. spaces is optional — an empty array means the project's main space. recordHistory applies to audit only.

Re-audit takes a three-field body, everything optional:

{ "projectName": null, "spaces": null, "notifySlack": false }

notifySlack opts in to a notification on the project's configured Slack webhook when the recorded run carries errors or warnings — like recording, an explicit ask, default false.

Generate takes:

{ "typeNames": [], "targetVersion": null, "extra": {}, "spaces": null }

typeNames restricts generation to named types (empty means everything eligible). targetVersion and extra exist for connectors with dialect switches; the Glue generator reads neither, so for this vendor they are inert.

Discovery

curl -sS -H "Authorization: Bearer $TOKEN" \
  "https://coremodels.example.com/graph/integrations/vendors"

Any authenticated user may call it. The Glue entry is your capability contract:

{
  "key": "glue",
  "displayName": "AWS Glue Data Catalog",
  "capabilities": "Import, Audit, Generate",
  "artifacts": {
    "tables": "required — `aws glue get-tables` JSON (one response, an array of responses, or a bare TableList)"
  }
}

Glue declares all three of Import, Audit and Generate. (A fourth flag, live sync, exists in the capability vocabulary and is deliberately not claimed by any connector — CoreModels never holds your cloud credentials.)

Import — Admin

curl -sS -X POST \
  "https://coremodels.example.com/graph/integrations/glue/import/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  --data-binary @import-request.json

Import is additive by design: tables become governed Types, columns and partition keys become Elements, vendor metadata is attached — and an already-governed node is never mutated or deleted. Re-import a grown catalog and the new tables land while the familiar ones are counted under datasetsSkippedExisting; new columns on already-governed tables land under fieldsAdded. Drift is not applied here. Drift is the audit's job, and acting on it is a human decision.

The response carries datasetsAdded, datasetsSkippedExisting, fieldsAdded, lineageEdgesAdded, lineageEdgesSkipped, nodesEnriched, a snapshotStored flag, and the lossiness / errors arrays. Errors mean could not proceed; lossiness means proceeded, and here is what was approximated.

Audit — Viewer

curl -sS -X POST \
  "https://coremodels.example.com/graph/integrations/glue/audit/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  --data-binary @audit-request.json

Read-only. recordHistory: true is the one opt-in write it will make, appending a compact record of the run to the project's rolling trail as bookkeeping — never as governed meaning. The response reports errorCount (greater than zero means governed meaning is violated), warningCount, infoCount, a codes map of finding code to count, driftedObjects, fingerprint (a content hash of the artifact), metrics, the itemized findings, a markdown rendering, historyRecorded, and lossiness.

Re-audit — Viewer

curl -sS -X POST \
  "https://coremodels.example.com/graph/integrations/glue/reaudit/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{}'

Same engine, different input. The live audit asks "do these fresh artifacts still conform to the governed model?"; re-audit asks "does the governed model still match the last-known estate?" by replaying the audit over the catalog snapshot stored at import time. No artifacts, no AWS access. Pass projectName to target a specific vendor-side database when several have been imported; null uses the most recently stored snapshot. Unlike audit, re-audit always records its run, and the report carries an extra Snapshot stored metric telling you how old that snapshot is.

If no snapshot exists, the call fails with a precise error: No stored estate snapshot for vendor 'glue' — import the vendor project first (imports persist the parsed snapshot).

History — Viewer

curl -sS -H "Authorization: Bearer $TOKEN" \
  "https://coremodels.example.com/graph/integrations/glue/history/$PROJECT_ID"

{
  "success": true,
  "vendor": "glue",
  "projects": [
    {
      "projectName": "lake",
      "runs": [
        {
          "at": "2026-08-04T09:41:07.6620114+00:00",
          "trigger": "ci",
          "errorCount": 0,
          "warningCount": 1,
          "infoCount": 2,
          "codes": { "semi-structured-column": 1, "table-no-description": 1, "classification-missing": 1 },
          "fingerprint": "3f9c1d5a7b2e4086"
        }
      ]
    }
  ]
}

Runs are newest first, grouped per vendor-side database, and the window is capped — the trail is a trend line, not an archive. trigger is one of audit, ci, reaudit, or scheduled.

Badge — Viewer

curl -sS -H "Authorization: Bearer $TOKEN" \
  "https://coremodels.example.com/graph/integrations/glue/badge/$PROJECT_ID" > glue-audit.svg

Returns image/svg+xml: a shields-style badge labeled glue audit, rendered from the latest recorded run. Green is clean, yellow is warnings only, red is errors, gray means no recorded runs yet (you will also get gray for an unrecognized vendor key rather than an error page). A project whose audits never set recordHistory keeps a gray badge forever.

Generate — Viewer

Glue claims the Generate capability, and the artifact is worth having:

curl -sS -X POST \
  "https://coremodels.example.com/graph/integrations/glue/generate/$PROJECT_ID" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{ "typeNames": [] }'

{
  "success": true,
  "artifacts": [
    { "name": "coremodels_glue_tables.sql", "kind": "sql", "content": "-- Generated by CoreModels …" }
  ],
  "lossiness": [],
  "errors": []
}

The content is Athena/Hive DDL built from the governed model:

-- Generated by CoreModels — governed Athena/Glue table definitions.
-- Set the LOCATION per table before running; meaning changes belong in CoreModels.

CREATE EXTERNAL TABLE IF NOT EXISTS events (
    event_id string COMMENT 'Event key.',
    user_id bigint,
    occurred_at timestamp,
    payload struct<action:string,value:double> COMMENT 'Raw payload.'
)
COMMENT 'Raw product events landed from Kinesis.'
PARTITIONED BY (
    event_date date COMMENT 'Partition.'
)
STORED AS PARQUET
LOCATION 's3://<your-bucket>/events/';

Note what rides along: the recorded Hive type is re-emitted verbatim (a struct<…> column comes back as a struct<…> column, not as a degraded string), governed descriptions and taxonomy allowed-values become column COMMENTs, partition-flagged elements are reconstructed into PARTITIONED BY, and the LOCATION is a placeholder you must set — the governed model knows your schema, not your bucket layout.

Two skips are declared rather than silent: governed views (Views are derived objects; DDL generation covers tables only.) and governed types with no elements, both reported as StructuralDrop records in lossiness. If nothing is eligible at all, the call returns success: false with No eligible tables found to generate DDL for.

Status — Viewer

curl -sS -H "Authorization: Bearer $TOKEN" \
  "https://coremodels.example.com/graph/integrations/glue/status/$PROJECT_ID"

{
  "success": true,
  "vendor": "glue",
  "imported": true,
  "state": {
    "vendor": "glue",
    "projectName": "lake",
    "importedAt": "2026-08-04T09:14:22.1043117+00:00",
    "toolVersion": null,
    "artifactVersion": null,
    "generatedAt": null,
    "sourceFingerprint": "3f9c1d5a7b2e4086",
    "counts": "fieldsAdded=0, lineageAdded=0, lineageSkipped=0, nodesEnriched=10",
    "facts": "{\"tables\":\"2\",\"views\":\"0\"}"
  },
  "governedDatasets": 2
}

imported: false with a null state simply means no Glue import has happened in this project. The tool-version, artifact-version and generated-at slots of the state record stay empty for Glue: a get-tables response carries no generator stamp, and we do not fabricate one. facts is the connector's own parser summary — how many tables and how many views it saw.

The machine-to-machine surface

curl -sS -X POST \
  "https://coremodels.example.com/v1/$PROJECT_ID/integrations/glue/audit" \
  -H "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" \
  --data-binary @audit-request.json

Identical request body, one structural difference in the response: it is wrapped in the standard API envelope, so every report field lives under data.*. A pipeline checks data.errorCount, not errorCount. When recordHistory is true here the run is recorded with trigger ci, which keeps automated runs distinguishable from interactive ones in the trail. The badge is available on this surface too, at GET v1/{projectId}/integrations/glue/badge.

What is not here is as important: reaudit and history live on the interactive surface only.

Failure modes worth handling

An unknown vendor key on any route returns success: false with a message that names the alternatives: Unknown vendor 'gluu'. Registered: …. A missing or empty artifact bag returns Body must include 'artifacts': { "<name>": "<content>" } (e.g. manifest for dbt). A payload that parses as JSON but is not shaped like get-tables output fails at the connector with The artifact is valid JSON but not shaped like get-tables output: …, and one with no named tables with No tables found (expected TableList entries).

The posture behind the whole table at the top: import is the only verb that writes governed structure, and even it only adds; audit and generate write nothing at all. That is why a Viewer-scoped key is all your automation ever needs. (One further Admin verb sits on the shared surface — POST graph/integrations/reconcile/{projectId} — which links datasets that two vendor estates govern as the same physical relation; it becomes interesting once Glue is not your only connected estate.)

For a task-shaped walk through the same routes, see the AWS Glue quickstart in the CoreModels documentation.

← Back to AWS Glue