Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

`surrogate_key` macro is not adding the || with concat in dbt Version > 0.20.1

See original GitHub issue

Describe the bug

With the dbt upgrade to v1.0.0/1.0.4/1.0.5 surrogate key macro is not generating the appropriate sql code.

Steps to reproduce

(1) Create a model with surrogate key macro for more than two columns. (2) dbt run will fail because of missing ‘||’ operator inside the concat statements.

Expected results

Properly formatted sql is expected with || symbol inside the concat statement.

Actual results

Screenshots and log output

Added above

System information

The contents of your packages.yml file: packages:

package: calogica/dbt_expectations version: [“>=0.5.0”, “<0.6.0”]

Which database are you using dbt with?

postgres
redshift
bigquery
snowflake
other (specify: ____________)

The output of dbt --version:

1.0.4

Additional context

Are you interested in contributing the fix?

Issue Analytics

State:
Created a year ago
Comments:7 (3 by maintainers)

Top GitHub Comments

1reaction

judahrandcommented, May 18, 2022

Though, actually, this looks like the new SQL will just fail - right?

1reaction

judahrandcommented, May 18, 2022

This seems like it is actually quite a significant issue…

I’d imagine a common use case for surrogate_key is to produce a key for an incremental model (since before 1.1.0 you could only use a single column). If the way this key is calculated has changed and people don’t do a ‘full-refresh’ on their incremental models then they’ll end up with duplicate data! In some cases a ‘full-refresh’ might not even be that feasible.

I think this should be fixed ASAP and the community notified of the potential issue.

Top Results From Across the Web

Upgrading to dbt utils v1.0 - dbt Developer Hub

Cause: No macro called MACRO_NAME exists. This is most likely because the macro has moved to the dbt namespace (see above). It could...

Generating Surrogate Keys Across Warehouses

The primary annoyance when creating surrogate keys comes when you try and concatenate a row that has a null value for one or...

Combing dbtutils "surrogate key" with "get filtered columns in ...

Try using the get_filtered_columns_in_relation macro to create a variable which you pass into the surrogate_key macro. There's no reason it ...

What is a surrogate key in database table? - dbt Developer Hub

A surrogate key is a unique identifier derived from the data itself. ... in-place to ensure your primary keys are unique and not...

Surrogate keys in dbt: Integers or hashes? | dbt Developer Blog

on their data models. Having a column that uniquely represents each entity helps ensure your data model is complete, does not contain duplicates ......