Semantic Layers Explained: What dbt Got Right and What It Still Can’t Do Alone
dbt changed analytics engineering. That's not a marketing line — it's the honest assessment of most practitioners who worked in the space before and after it existed. The ability to define transformations in SQL, version them in git, document them in YAML, and test them in CI turned what used to be an artisanal craft of ad-hoc warehouse scripts into something resembling a real software engineering discipline. We use dbt. We recommend dbt. And we've spent two years watching the specific places where it runs out of surface area — and what happens to teams when they try to fill that gap manually.
What dbt actually solved
Before getting into the gaps, it's worth being specific about what dbt got right. Three things stand out.
First, it made SQL transformation composable. The ref() macro and the DAG-based dependency model meant you could build transformation chains where each model depended on upstream outputs cleanly, and the framework handled execution order. This was not trivial. Before ref(), analytics engineers maintained ordering manually or relied on brittle orchestration scripts.
Second, it introduced testable data quality. The generic tests — not_null, unique, accepted_values, relationships — gave teams a lightweight contract enforcement layer. Running dbt test in CI before merging model changes became the analytics equivalent of a unit test suite. Not as rigorous as a real test framework, but structurally similar and dramatically better than nothing.
Third, it created documentation as a first-class output. dbt docs generate produces a browsable lineage graph with column-level descriptions. For teams that actually maintain their schema.yml files, this is genuinely useful. It's the closest most analytics teams get to a data catalog without buying a separate product.
Where dbt stops and the gap begins
dbt is a transformation framework. It is very good at taking data from one shape and producing it in another. What it doesn't do — by design, not by oversight — is manage the semantic contract between upstream producers and downstream consumers.
Metric definitions are SQL, not declarations
dbt metrics (now part of MetricFlow) lets you define metrics in YAML. This is a meaningful improvement over ad-hoc calculated fields in Looker explores. But metric definitions in dbt are still tightly coupled to the underlying model SQL. When the underlying tables change, the metric definitions don't automatically update, flag the impact, or notify an owner. The lineage awareness lives at the model level; the semantic meaning lives only in the head of the person who wrote the YAML.
No schema change propagation awareness
When your Snowflake source table adds a column, removes a column, or changes a data type, dbt has no mechanism to detect this, assess which downstream models are affected, and route a structured review to metric owners. You can write dbt source freshness tests, but these require you to already know what to test for. The unknown unknowns — the schema changes you didn't anticipate — pass through silently until a model fails or produces wrong numbers.
dbt docs go stale almost immediately
In practice, dbt docs are excellent at the moment they're written and progressively less reliable after that. Engineers add columns to models faster than they update schema.yml files. Description fields accumulate stale references to old business logic. The lineage graph shows what the SQL does, not what the business intends the metric to mean. After six months of active development, a non-trivial fraction of metric descriptions in most dbt projects are inaccurate.
The semantic gap between tables and business meaning
This is the fundamental limitation. dbt knows that fct_revenue.net_arr is a decimal column in the finance schema. It does not know that net_arr means "annualized recurring revenue net of expansion, contraction, and churn as defined by the finance team after the Q3 pricing model change." That definition lives in a Google Doc, a Confluence page, a Slack thread, or nobody's head. When someone asks "what does this metric mean?" the answer requires a human conversation, not a tool lookup.
The abstraction stack you actually need
A complete semantic layer for an analytics team running 3-8 sources with 2-4 engineers needs to operate at three levels that dbt covers only partially:
| Layer | What it does | dbt coverage |
|---|---|---|
| Transformation layer | Moves data from source schemas to analytics-ready models | Strong (core dbt capability) |
| Contract layer | Enforces column-level contracts between producers and consumers; blocks promotions on violations | Partial (dbt tests exist but don't block on schema change detection) |
| Semantic catalog layer | Publishes canonical metric definitions, owner assignments, derivation logic, freshness SLAs as a queryable API | Minimal (dbt docs are static, not machine-queryable in real time) |
The team that tried to build the full stack in dbt
We've spoken with analytics teams that attempted to build a complete semantic layer using only dbt — typically through a combination of dbt metrics, extensive schema.yml documentation, and custom macros for lineage tracking. The results are instructive.
The metric YAML files become very long very fast. Maintaining them accurately requires discipline that erodes under sprint pressure. The custom macros for detecting schema changes become a maintenance burden of their own. The "catalog" is a static HTML export that goes stale between quarterly documentation sprints. The system works well for a team of two with strong internal discipline; it degrades visibly at three or four engineers with mixed tenure.
dbt gives you a very good foundation for the transformation layer. The semantic and contract layers on top of it require either significant custom investment or a purpose-built tool — and the custom investment is itself a form of the pipeline maintenance burden you were trying to escape.
What a native semantic layer integration looks like
The pattern that works: a semantic layer that reads dbt model definitions via the dbt Core or dbt Cloud APIs, understands the lineage graph, and extends it with canonical metric definitions that are machine-queryable, owner-assigned, and connected to freshness SLAs and contract checks.
This means your dbt models continue to be the authoritative SQL transformation layer. You don't rewrite them. What changes is that above the dbt layer, there's a governed catalog where every metric has an explicit business definition, an owner, a derivation path back to the dbt model, and a freshness expectation that's monitored automatically.
When an upstream schema changes, the system knows which dbt models depend on the affected columns, which catalog metrics depend on those models, who owns each metric, and what the expected freshness interval is. That chain of information — from column to metric to owner to SLA — is what turns a 12-hour debugging session into a 45-minute structured review.
dbt is a necessary part of a good analytics stack. The semantic layer above it is what most teams build manually, badly, and then rebuild from scratch every time the team grows. That pattern has a better answer, and it starts with understanding precisely where dbt's scope ends and where a catalog-and-contract layer needs to begin.
