Identity, Types and Honest Loss: What a Fabric Import Actually Writes

Governance starts with a question that sounds trivial and is not: what is a table, once it stops being a table?

In CoreModels a Microsoft Fabric table becomes a governed Type, its columns become Elements, and its constraints become checks and references. But the fact that makes the whole loop work is none of those. It is the identity: every imported node carries a mapsTo assertion pairing the standard fabric with the estate identity catalog.schema.table. That assertion is what lets an audit six months later say "this governed node and that warehouse column are the same thing" after somebody has renamed the node, moved it, or restructured the model around it.

This is the full mapping — what lands where, what rides metadata rather than meaning, which rules the audit applies, and where the connector is honestly lossy. It applies identically to SQL Server, since the artifact contract is ANSI T-SQL INFORMATION_SCHEMA.

Parse first, write second

The connector itself never touches the graph. It parses artifacts into a neutral estate model — datasets, fields, checks, lineage edges — and contributes audit rules and a generator. Everything graph-shaped is done by vendor-neutral services on the other side of that seam.

Parsing is deliberately tolerant. Rows without a TABLE_NAME are skipped. A TABLE_TYPE containing VIEW marks the dataset as a view; anything else is a table. The dataset identity is built by joining the non-empty parts of catalog, schema and table with dots — which is the first practical consequence of the extraction query: omit TABLE_CATALOG and every identity in the project becomes schema.table instead, changing what later matching and drift scoping can see.

Only unusable payloads fail. Not JSON, no rows, no TABLE_NAME values anywhere — those come back as errors naming the artifact. Everything else that is odd comes back as lossiness on a successful parse.

Datasets, fields, ids

A dataset becomes a Type whose id is the sanitized identity and whose label is the bare table name. Sanitization folds separators away and camel-cases what follows, so wh_sales.dbo.orders becomes the node id whSalesDboOrders. A field's id is the dataset id plus the Pascal-cased column name: whSalesDboOrdersOrderTotal.

The sanitization is not cosmetic. Node ids must be alphanumeric because the id index tokenizes on separators like . and _, so a raw identity could never be matched by the exact-id lookup that follows every write. The original identity is never lost — it lives on the node label, on the mapsTo assertion, and in the vendor metadata.

Sanitization can collide, and we made the import refuse rather than merge. If two distinct estate identities collapse onto one graph id, the import fails with an error naming both: "Distinct vendor identities collapse onto the same graph id after sanitization ... Rename one of them in the vendor estate." Silently merging two tables would be the worst possible outcome, so it does not happen.

Types

Native types are reconstructed before mapping, so the estate's own spelling survives: CHARACTER_MAXIMUM_LENGTH rebuilds varchar(50), a length of -1 becomes nvarchar(max), and precision and scale rebuild decimal(18,2). That reconstructed string is what gets recorded, and what drift compares against later.

The IR mapping runs on the type family:

T-SQL family	Governed primitive	Approximated
int, bigint, smallint, tinyint	Integer	no
decimal(p,0), numeric(p,0)	Integer	yes — the precision bound is dropped
decimal(p,s), numeric(p,s) with s > 0	Double	yes — exact decimal to binary double
float, real	Double	no
bit	Boolean	no
date, datetime, datetime2, smalldatetime, datetimeoffset	DateTime	no
time	String	yes — time-of-day has no primitive slot
varchar(n), nvarchar(n), char(n), nchar(n)	String	yes — the declared length is dropped
text	String	no
everything else	String	yes

bit is the connector's one override on the shared warehouse mapping; everything else rides the common map. That last row is where the honesty lives: uniqueidentifier, varbinary, money, xml, ntext and friends all land as String with a type-approximation record explaining that the exact native type was preserved in the metadata rather than in the structure. One quirk worth flagging: timestamp, which in T-SQL is the deprecated synonym for rowversion rather than a clock value, matches the date-time family by name and maps to DateTime. The native token is recorded verbatim either way.

Checks, keys and references

IS_NULLABLE = NO becomes a NotNull check. Everything else comes from the optional keys artifact.

Primary keys are grouped by constraint name, which is what makes composite keys behave. A single-column key gives its column both NotNull and Unique. A composite key gives every participating column NotNull and records the key itself as a dataset-level custom check named composite_primary_key, whose detail is the comma-joined column list — deliberately not a per-column uniqueness claim, because no individual column is unique.

A foreign-key row adds a relationship check and a reference to the target identity. When the target table is in the same extract, the reference becomes a real expected-type relation between the two governed Types. When it is not — a cross-schema key you did not extract — the field is written as a plain value and the import reports a semantic-narrowing record saying exactly which target pointed outside the snapshot.

Two silent behaviors are worth knowing. Key rows whose table or column cannot be resolved against the extract are skipped, so a keys file from a different schema than the information_schema file contributes nothing. And within one dataset, the first row for a given column name wins; later duplicates are ignored.

Taxonomies never appear from this connector. INFORMATION_SCHEMA declares no accepted-value sets, so there is nothing to build a controlled list from — and no lineage, either, which is why lineageEdgesAdded is always zero and the shared dependency relation group stays empty for a Fabric-only project.

What rides metadata instead of meaning

Every imported node gets one value on a mixin type named Microsoft Fabric Metadata. Property ids are the vendor key plus the property name, so a column's native type lands under fabricDataType, its checks under fabricChecks, a table's physical name under fabricRelationName, its materialization under fabricMaterialization, its identity under fabricUniqueId.

The checks value is JSON:

[{"kind":"NotNull","name":"not_null","severity":null,"detail":null},
 {"kind":"Unique","name":"primary_key","severity":null,"detail":null}]

These values are refreshed on every import. That is the deliberate asymmetry of the whole design: governed structure is additive and never mutated, because changing meaning is a human act, while vendor metadata mirrors the estate and is expected to move. Every property is rewritten on each refresh — including as empty when a fact disappears — so stale metadata cannot linger.

Nullability sits in an interesting place. The governed graph has no enforced required flag; the element-level fact is stored and visible but not enforced by the graph itself. Which is why DDL generation reads not-null from the metadata checks rather than from the structure — and why the audit compares native type strings rather than primitives.

Three bookkeeping nodes complete the picture, one per vendor and estate: an integration state node recording the last import, an estate snapshot node holding the parsed estate compressed and encoded, and an audit history node holding the rolling run trail. None of them is governed meaning; all of them are what make the artifact-free re-audit, the status call and the drift trail possible.

The rules the audit applies

Findings land in three sections with stable kebab-case codes.

Coverage reports what the estate has and the graph does not: dataset-unmapped (Warning) for a table nobody imported, field-unmapped (Info) for a column added since the last import.

Drift reports disagreement. dataset-removed and field-removed are Errors — governed nodes with no counterpart left in the estate. field-type-drift is an Error, raised when the recorded native type and the extracted one differ after whitespace and case normalization, with both values in the finding detail. contract-drift and the enum codes exist for connectors whose estates declare those things.

Conformance carries the two Fabric rules. key-column-undeclared (Warning) fires on a table column named id, or ending in _id or Id, that carries no uniqueness, relationship or reference declaration — never on views. And descriptions-not-extracted (Info) is emitted once per audit, in the Coverage section, because T-SQL INFORMATION_SCHEMA has no descriptions: they live in extended properties, which this extract does not cover.

Two comparison subtleties are worth internalizing. dataset-removed only fires within the namespaces present in the extract being audited, so a project governing several warehouses does not report one estate as deleted while auditing another. And field-removed matches on vendor identity, never on labels — relabeling a governed element is a normal modeling act and must not read as estate drift.

There is one predictable false positive. If a modeller governs a column with a taxonomy — a real status list, say — every Fabric audit will report enum-constraint-removed on it, because the estate cannot declare accepted values through INFORMATION_SCHEMA. That is the audit working as designed on a signal the artifact cannot carry, and it is worth knowing before you gate on warnings.

The limits, stated plainly

No live connection: CoreModels never holds Fabric or SQL Server credentials, and there is no polling agent. No descriptions, as above. No lineage. When the encoded estate snapshot exceeds the storage cap, the import says snapshotStored: false with a lossiness record — fresh-artifact audits keep working, the artifact-free re-audit does not.

Generation has its own edges: views are skipped with a declared record, a governed type with no elements is skipped, a table whose key is composite gets columns but no PRIMARY KEY line, and foreign keys are emitted as FOREIGN KEY ([column]) REFERENCES <table> without a target column list. The skips are declared as lossiness records; the rest is visible in the emitted script itself.

And when the same warehouse is also described by a transformation tool, reconciliation links the two estates on physical name — catalog.schema.table — with reciprocal sameAs assertions, so one physical table stops being two governed things.

Everything described here starts from two read-only queries, both documented in the Microsoft Fabric quickstart.