Move explore configuration to dbt exposures

The next dbt release introduces meta to dbt exposures: https://github.com/dbt-labs/dbt/releases/tag/v0.20.0rc1 https://github.com/dbt-labs/dbt/pull/3405

What is an exposure?

An exposure doesn’t have much meaning in dbt beyond providing lineage and a link from dbt docs to the “exposure”: https://docs.getdbt.com/docs/building-a-dbt-project/exposures

A notable feature of exposures, is that they can reference multiple tables, providing the lineage into that exposure node.

Why move to exposures?

Currently we add sql join logic on to a single model, which is messy because it specifies joins to other tables that might not necessarily exist. dbt exposures makes explicit the tables necessary for that exposure to be valid. So I think it makes much more sense to define the joins here.

We have an outstanding issue #121 to figure out lineage of joins, we’d get that for free with exposures.

We have another issue #170 to hide certain models in the UI, exposures would also be helpful here

It also teases out the distinct entities: an explore is made of one or more models. At the moment it’s a bit confusing that a model is an explore but also might join to another model in another explore.

What if I want my users dbt model as the base for two different explores?! That’s totally reasonable and impossible today.

The future

Finally there is a lot of future configuration that really doesn’t live at the model level. See the options for a looker explore: https://docs.looker.com/reference/explore-reference

e.g.

• access - who can see this explore
• renaming - base tables might be renamed to something more sensible depending on the context of the explore
• default filters - restricting the entire explore to the last 6 months of data for example: https://docs.looker.com/reference/explore-params/sql_always_where
• anything related to join logic - hiding metrics that don’t make sense, handling symmetric aggregates, etc.

Example

exposures:
  
  - name: jaffle_metrics # becomes name of explore (doesn't have to be the base model name)
    type: dashboard  # dbt specific - dashboard|notebook|analysis|ml|application
    url: https://bi.tool/dashboards/1 # this is optional, ideally we need to pass this information back :(
    description: >
      Did someone say "exponential growth"?   # becomes the description of the explore
    owner: [e-mail]   # This is required, not that useful to us (unless we map it to a user in lightdash in future ❤️ )
    
    depends_on:                  # These tables are available as joins
      - ref('fct_orders')
      - ref('dim_customers')
      - source('gsheets', 'goals')    # unsure if we should support sources
      
    meta:
      base: ref('fct_orders')
      joins:
        - model: ref('dim_customers')
           sql_on: ${fct_orders}.customer_id = ${dim_customers}.id

(mixing dbt references and lightdash references looks very messy here, need to think about that)

[TuringLovesDeathMetal]

ohhhhhhhh I see what you’re saying. So basically, every model is a “model” in Lightdash, but then you have “Explores” which are built from model(s) but can be multiple models joined together or a subset of data from a model (e.g. with a filter).

This actually makes a lot of sense! I really like it!

[stale[bot]]

Is this issue still relevant? There have been no updates for 60 days, please close the issue or keep the conversation going!