dbt — Interview Q&A
Q: What problem does dbt actually solve that raw SQL scripts don't?
A: Three things together: dependency management (via ref()/source(), dbt automatically figures out the correct execution order across models, rather than someone manually sequencing SQL scripts and having to remember to update that order every time a dependency changes), testability (generic and custom tests run as real queries against built data, turning "we hope the transformation is correct" into an actual automated check), and lineage/documentation (automatically derived from the same ref()/source() calls already in the code, so it can't drift out of sync with reality the way a hand-maintained architecture diagram can). Raw SQL scripts can technically do the same transformations, but none of that structure comes for free — it has to be built and maintained separately, and typically isn't.
Q: Explain the difference between ref() and source(), and why that distinction matters.
A: source() marks a table dbt doesn't create — the actual raw entry point of data into the warehouse. ref() references another dbt model by its logical name, not its actual physical table name. This distinction is what makes the dependency graph meaningful: dbt uses every ref()/source() call across the project to build the DAG and determine correct execution order automatically. It also means a model's underlying table name can change without breaking anything downstream, since every reference goes through the logical ref() name rather than a hardcoded physical table name.
Q: When would you use incremental materialization instead of table, and what's the most common bug when implementing it?
A: Once a table is large enough that a full rebuild on every dbt run is genuinely too slow — incremental materialization processes only new or changed rows instead of reprocessing full history every time. The most common bug is a missing or incorrect unique_key — without a correct unique key, dbt's incremental logic simply appends "new" rows matching the incremental filter, including rows that are actually updates to existing records, producing silent duplicate rows rather than merges. This tends to go unnoticed for a while because it doesn't produce an error, just slowly-growing, subtly wrong data.
Q: How would you handle a genuine business-logic validation that dbt's built-in generic tests (unique, not_null, accepted_values) can't express?
A: A singular test — a plain SQL file that should return zero rows if the data is valid, placed in the tests/ directory. This lets you encode arbitrary domain-specific assertions, like "an order total should never be negative" or "a customer's lifetime value should never exceed the sum of their order totals" — logic specific to your actual data's business rules that generic structural tests have no way to express. Writing one of these each time a real data quality bug is discovered turns that one-off bug into a permanent regression test against the same class of issue recurring silently later.
Q: What's Slim CI, and why does it matter for a large dbt project?
A: It's dbt's state-based selection (--select state:modified+), which compares the current PR against a known-good production state artifact and runs only the models that changed plus everything downstream of them — not the entire project. For a project with hundreds of models, running the full project on every single PR becomes impractically slow and expensive; Slim CI keeps CI runtime proportional to the actual size of a change rather than the size of the whole project, which is essential once a project reaches real scale.
Q: What's an exposure in dbt, and what problem does it solve?
A: An exposure declares that something outside dbt — a BI dashboard, a downstream ML pipeline, a reverse-ETL sync — depends on specific dbt models. Without exposures, a model's actual downstream business consumers are invisible to dbt and to anyone reviewing a proposed change; with them declared, the lineage graph extends past dbt's own models to show real business impact — "changing this column would break the executive revenue dashboard" becomes a documented, visible fact in the lineage graph instead of tribal knowledge only one person happens to remember.
Q: How would you decide between dbt Core and dbt Cloud for a new project?
A: They run the same dbt project and produce identical models — the difference is entirely about who owns the operational overhead of scheduling, CI integration, and documentation hosting. dbt Core is free but self-orchestrated — you build scheduling and CI integration yourself, typically via a general-purpose orchestrator like Airflow or Dagster. dbt Cloud is a managed commercial offering with hosted scheduling, CI, and docs, at real licensing cost. A team already running Airflow or Dagster for other pipelines often finds Core the more natural fit, since the orchestration infrastructure already exists; a team without that infrastructure often finds Cloud's managed scheduling a faster path to a working production pipeline without building that infrastructure first.