Seven Problems, One Cause
If you run dbt at scale, you have probably hit several of these. They arrive as unrelated annoyances, get filed under different headings, and get fixed — when they get fixed — by different people using different tools.
Seven Problems, One Cause
If you run dbt at scale, you have probably hit several of these. They arrive as unrelated annoyances, get filed under different headings, and get fixed — when they get fixed — by different people using different tools.
They are the same problem wearing seven coats.
The seven
1. Two files test the same column against different value lists. Both pass. Someone added a status six months ago and updated the model they were working in. The other files still carry the old list. No test fails, because each list is internally valid — dbt has no object that spans them. (Eleven Copies of One Enum)
2. The rename you have been postponing for eight months.
status should be order_status. Everyone agrees. Nobody can tell you what breaks, because
dbt ls --select model+ answers a narrower question than the one you are asking. So it stays on the
list. (The Rename You Didn't Make)
3. The pull request everything passed.
Delete an accepted_values test, open a PR — green across the board. dbt build runs the tests in
the branch, and a deleted test is not a failing test. (Every Check You Run Compares the PR to the
Last Commit)
4. "Ask in #analytics and see who replies." Someone wants to add a value to a status column and cannot find who decides. Two people answer, one of them hedging. Six months later a number is wrong and the retrospective concludes we should have a process. (Ask in #analytics and See Who Replies)
5. The agent that guessed.
Your text-to-SQL tool filters on status = 'returned' and undercounts by everything sitting in
return_pending. It could not have known — it had the column name and a description that restates
the column name. (The Fourth Option for a Column Named status)
6. The column comment that says status is the status.
Someone querying that table at 11pm has no way to learn what the values mean without reading your
SQL or finding you. (The Ticket Nobody Files)
7. The consumers who cannot read your contract. Your mart has an enforced dbt contract. The reverse-ETL job, the ML pipeline and the team down the hall cannot parse it, so what they rely on is an agreement nobody wrote down — and nobody keeps a list of who they are. (The Consumers dbt Can't See)
What they have in common
Every system in your stack stores shapes. None of them stores meaning.
dbt stores a column's type, its tests, and free text. The warehouse stores a type and a nullability flag. The catalog stores a copy of both that starts decaying the moment it is written. Not one of them has a place to record what the value actually is — which values are permitted and what each one means, what this concept is called outside your company, and who gets to decide.
So meaning gets re-typed as strings into each system that needs it. Eleven times. And then it drifts, because eleven copies of anything drift.
That is the cause. Every one of the seven is a symptom of it:
| Symptom | What is missing |
|---|---|
| Lists that disagree | Nowhere to define permitted values once |
| Rename you cannot risk | Nowhere that records what depends on a concept |
| PR that passes | Nothing to compare a change against |
| Nobody decides | Nowhere to record who owns a definition |
| Agent guesses | Nothing machine-readable to look meaning up in |
| Useless column comment | No path from a definition to the warehouse |
| Contract outsiders can't read | Only one output format, and it stops at dbt |
Why the usual fixes do not hold
Three things get tried, and each fails for the same structural reason.
More tests. Tests check data against rules. They do not check whether two rules agree with each other, and they cannot, because there is no object that holds "the rule" independently of the eleven places it is written.
More process. A second reviewer, a checklist, a PR template — each asks a human to hold the prior agreement in their head at review time. That fails not because people are careless but because the agreement is not an artifact. There is nothing to compare against, even for someone paying full attention.
A catalog. Catalogs are good at discovery: what exists, where it lives, roughly what it connects to. But a catalog describes the model from outside. It cannot be the place a definition lives, because nothing downstream is generated from it — so it decays silently while continuing to look authoritative.
The common failure: all three attach to files, and meaning is not file-shaped. One concept lives in eleven files at once. Anything that attaches to a file governs one-eleventh of it.
What a home for meaning actually looks like
Four concrete things, none of them abstract:
A vocabulary instead of a string array. The permitted values become an object with a name, a
description per value, and structure where the concept has it — return_pending sitting under
returned rather than beside it. Now the question "do these two lists agree?" has something to
compare.
An identity outside your company. A column can be bound to a public term, so "customer" means something checkable rather than something local.
An owner, and a forum. Who decides what this may contain, and where that decision gets made. A forum rather than a person, because people change teams and the register should not quietly become wrong while still looking authoritative.
Publication. This is the part that makes the rest hold. Everything else becomes an output: your dbt property files, a JSON Schema for the API team, an ODCS contract for a partner, the comment in the warehouse column. Generated, not maintained.
That last one is why this does not become another catalog. A definition that generates artifacts cannot silently drift from them — regenerate and they match, because there is nothing else for them to match.
The one mechanism worth knowing
Of everything here, this is the detail that changes what a non-dbt person experiences.
persist_docs pushes a dbt column description into the actual column comment in your warehouse.
The description is the only field that makes that trip. So a bound term appended there reaches
somebody querying the table who has never heard of your modeling tool, has no access to your repo,
and is not going to ask you at 11pm.
persist_docs is off by default. If your project has never set it, turning it on is part of the
work, and adapter support for column comments varies — worth checking before you promise anybody the
definition will show up in the warehouse.
Meaning authored once, ending up enforced in the warehouse, readable by a human, and resolvable by a machine. That sentence is the whole product.
What it costs, honestly
A morning, up front. Somebody has to decide what things mean. No tool does that part, and any tool claiming to is selling you a confident guess.
And it is worth naming what does not happen. We return files; your own pull request lands them.
We generate model contracts, not sources.yml or staging models — dbt's own codegen owns that
ground. We never connect to your warehouse and never hold credentials.
Success and lossless are two different words. Every generation hands back a report of what could
not cross the boundary — a hierarchical vocabulary flattening into dbt's flat accepted_values, a
relationships test omitted because the target could not be resolved. We would rather tell you than
quietly produce a weaker contract than you think you have. (Six Refusals)
Where to start
Pick by whichever of the seven bit you most recently.
| If this is your week | Start here |
|---|---|
| Your agent gets column meanings wrong | Grounding an Agent in Your dbt Project |
| You suspect your value lists have drifted | Finding the accepted_values List That Drifted |
| Nobody knows who decides definitions | Building an Ownership Register |
| A PR broke something review should have caught | dbt Contract Automation in CI |
| You want the whole loop end to end | Manifest In, Contracts Out |
Three of those work self-serve today. The ones that publish back into your repo or gate your CI are available with us alongside you while the self-serve path is finished — the recipe cards say which is which, and they are not vague about it.
The sentence to remember
Your definitions stop being something you maintain in eleven places, and start being something you publish.
Everything above is that one change, seen from seven angles.