Read-Only by Design: How We Earn Trust With Your Glue Catalog

"What does it touch?" is the first question a platform team asks about any tool that wants to look at their data lake, and it is the right question. Our answer for the AWS Glue integration is unusually short: nothing in your AWS account, ever. CoreModels never holds AWS credentials and never opens a connection to AWS. You run aws glue get-tables yourself, with your own credentials, inside your own perimeter, and upload the resulting JSON. A live-sync mode is a capability we have deliberately deferred rather than quietly shipped, precisely because "we'll be careful with your keys" is a weaker promise than "we never have them."

But credential posture is only the perimeter of trust. What makes a governance layer safe to depend on is how it behaves inside the boundary — what it writes, what it refuses to write, and what it admits it cannot represent. Five commitments govern the Glue integration, and each one is enforced in the design rather than in the documentation.

Import is additive — always

When you import a Glue catalog, tables become governed types and columns become elements — but only where nothing exists yet. Re-importing a catalog never mutates or deletes an existing governed node. If your steward has renamed a description, attached allowed values, or tightened a definition since the last import, a fresh import cannot silently undo that work; the response even reports how many datasets were skipped because they already existed. The one thing a re-import does refresh is vendor bookkeeping — the estate metadata that records what Glue currently says — because that is a fact about AWS, not a decision about meaning.

This is the split that makes the whole model coherent: the catalog owns what is, the governed model owns what should be, and import is only ever allowed to add to the second, never to overwrite it.

Meaning changes go through a human

So what happens when the catalog and the governed model disagree? Nothing automatic. Disagreement is exactly what the audit exists to surface — a retyped column becomes a field-type-drift finding showing both sides, a vanished table becomes dataset-removed — and a finding is evidence, not an instruction. No audit result ever modifies the governed model. Accepting a drift as the new truth is a deliberate act by a person with the authority to make it, performed in the governed model itself. We think of the audit as a witness with an excellent memory and no opinions about what you should do.

Lossiness is declared, not hidden

Glue's catalog and a governed semantic model do not have identical expressive power, and pretending otherwise would be its own form of dishonesty. So the integration keeps a lossiness channel alongside its success channel, and uses it. Hive struct<>, map<>, and array<> types cannot be faithfully carried into a flat element, so they are approximated as strings — and the approximation is recorded, and a dedicated audit rule keeps flagging those columns as carrying ungoverned inner schemas. decimal(p,s) maps to the nearest governed primitive and is marked as an approximation. And because Glue declares no primary keys and no nullability, CoreModels invents neither: a governance layer that fabricated constraints the source never stated would be manufacturing false confidence at the exact point where confidence matters most.

The same honesty applies to storage limits. Import persists a snapshot of your parsed catalog so it can be re-audited later; if a very large catalog exceeds the snapshot cap, the import says so outright — snapshotStored: false, with a lossiness record explaining the consequence — rather than truncating silently.

Drift is evidence, and evidence accumulates

A single audit is a photograph; governance needs a film. Every artifact carries a content fingerprint, every finding carries a stable code, a severity, and a subject, and runs can be recorded into a rolling per-project history. The check also runs in both directions: a live audit asks whether fresh catalog exports still conform to the governed model, while a re-audit replays the snapshot stored at import time against the current model — so a model change reveals its blast radius on demand, without anyone re-exporting anything.

Even the bookkeeping respects consent. The audit verb is strictly read-only by default; recording a run into the history happens only when the caller asks for it. (The re-audit verb always records — its entire purpose is the trail.) The status badge your README embeds is rendered from that recorded history: green, yellow, red, or gray for "no recorded runs," which is itself an honest state rather than a pretend-green.

Generated artifacts are proposals, not deployments

The loop closes with generation: CoreModels can emit Athena-ready CREATE EXTERNAL TABLE DDL from the governed model, with governed descriptions and allowed values riding the column comments and partition keys forming the PARTITIONED BY clause. And here the same philosophy holds. The generated file contains a placeholder S3 location that a human must set; it never executes anything; views are skipped with a declared reason rather than guessed at. Its header says, in so many words, that meaning changes belong in CoreModels. Generation gives you an artifact to review and deploy through your own process — it does not reach into your lake.

Trust as an architecture, not a promise

Run the five commitments together and a shape emerges: writes are additive, meaning is human-gated, approximation is confessed, drift is evidence with a chain of custody, and outputs stop at the review boundary. None of this depends on our good behavior at runtime, because the capabilities to misbehave were left out of the design. That is what we think "trustworthy" has to mean for infrastructure that sits next to the system of record for your entire lake.

The AWS Glue quickstart in the CoreModels docs spells out each call, each role it requires, and each thing that is deliberately not supported.