dbt exposures generated from TMDL

The question comes up at every change: if I drop this column, what breaks. The transformation tool answers for its own models, precisely, and stops at the last table. What reads it afterwards (a report, a measure, a dashboard opened every Monday by a management team) does not exist in its graph. The honest answer is therefore “I do not know”, and the answer given is “I do not think so”.

Exposures exist for exactly this: declaring what consumes a model without being produced by it. They have a deserved reputation for uselessness, and the reason is simple: written by hand, they are stale within a month. Somebody adds a measure, nobody edits the file, and the graph now lies with a graph’s authority. False lineage is worse than no lineage, because people act on it.

While the semantic model is a binary file sitting on somebody’s personal drive, there is nothing to parse and the discussion ends there. Now that it is written as text, it becomes a source like any other, and if it lives in the same repository as the transformation project, a script can read both in one pass. That is the only condition, and it is structural rather than technical.

warehouse/
├── dbt/
│   ├── models/marts/fct_sales.sql
│   └── models/marts/_marts.yml        # generated exposures land here
└── semantic/
    └── Sales.SemanticModel/
        └── definition/tables/Sales.tmdl

A measure carries everything needed: its display name, its expression, and whatever description its author bothered to write. The tables it queries are readable in the expression itself.

/// Revenue excluding tax, at invoice date. Excludes cancelled orders.
measure 'Revenue ex tax' =
    CALCULATE (
        SUMX ( Sales, Sales[quantity] * Sales[unit_price] ),
        Sales[status] <> "cancelled"
    )
    formatString: #,##0 €
    displayFolder: Measures\Sales

The script fits on a page: read the measures, find the tables named in the expression, write an exposures file. Two schema constraints decide the shape of the result, and discovering them at run time costs an hour each.

import re
from pathlib import Path

import yaml

MEASURE = re.compile(
    r"(?:^///(?P<doc>.*)$
)?"      # the /// description line, when there is one
    r"^s*measures+'(?P<name>[^']+)'s*=s*(?P<dax>.*?)(?=^s*(?:measure|table|Z))",
    re.M | re.S,
)

def slug(label: str) -> str:
    # An exposure name accepts letters, digits and underscores only, so a measure
    # called 'Revenue ex tax' cannot be one. The real name goes to label.
    return re.sub(r"[^a-z0-9]+", "_", label.lower()).strip("_")

def exposures(tmdl_dir: Path, models: set[str]) -> list[dict]:
    found = []
    for path in tmdl_dir.rglob("*.tmdl"):
        for m in MEASURE.finditer(path.read_text(encoding="utf-8")):
            dax = m.group("dax").strip()
            # Tables cited in the expression, kept only when dbt actually builds one
            # of that name — anything else is a calculated table, not a dependency.
            cited = {t.lower() for t in re.findall(r"([A-Za-z_]w*)s*[", dax)}
            depends = sorted(cited & models)
            if not depends:
                continue
            found.append({
                "name": slug(m.group("name")),
                "label": m.group("name"),
                "type": "dashboard",
                # description is markdown, which is the whole trick: a fenced block
                # puts the DAX itself into the generated documentation site.
                "description": f"{(m.group('doc') or '').strip()}

```dax
{dax}
```",
                "depends_on": [f"ref('{d}')" for d in depends],
                "config": {"meta": {"source_file": str(path)}},
            })
    return found

The first trap is that an exposure name accepts letters, digits and underscores only. A measure called “Revenue ex tax” therefore cannot carry its own name: it has to be transliterated, with the real name going to `label`, or generation fails on half the model.

The second is in fact the point of the exercise. The `description` field accepts markdown, therefore a code block, and that is how the DAX expression enters the generated documentation site. Nobody wrote anything; a measure’s documentation is its own definition, current by construction, next to the model that feeds it.

exposures:
  - name: revenue_ex_tax
    label: Revenue ex tax
    type: dashboard
    description: |
      Revenue excluding tax, at invoice date. Excludes cancelled orders.

      ```dax
      CALCULATE (
          SUMX ( Sales, Sales[quantity] * Sales[unit_price] ),
          Sales[status] <> "cancelled"
      )
      ```
    depends_on:
      - ref('fct_sales')
    config:
      meta:
        source_file: semantic/Sales.SemanticModel/definition/tables/Sales.tmdl

Generation runs in the integration chain and the produced file is versioned. The consequence is not the documentation, which is a pleasant side effect: it is that the pull request removing a column makes the disappearance of the exposures depending on it show up in its own diff. The question “what breaks” stops being asked in a meeting because it has already been answered in review.

Parsing the expression with a regular expression is crude and I own it: it finds the tables named, it does not understand DAX. A measure calling another without naming a table produces an exposure with no dependency, therefore nothing: the script skips them silently, and that is the flaw to watch. A proper parser exists, it costs a great deal more than a page, and I did not judge it worth it at this scale.

Nor do I say anything about the reverse direction, which would push model descriptions into the measures rather than the other way round. It is feasible and I distrust it: it would make a generated file the source of a reviewed one, and the day the two diverge nobody knows which one is authoritative.

• dbt · propriétés d’une exposure
• TMDL · format de définition d’un modèle tabulaire