question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Optionally skip `expand_target_column_types` during incremental materialization

See original GitHub issue

Describe the feature

Would like to be able to skip expand_target_column_types in an incremental materialization; we’d like the types in our tables to be immutable/fully specified by code.

Describe alternatives you’ve considered

Considered overriding the materialization and removing

{% do adapter.expand_target_column_types(
           from_relation=tmp_relation,
           to_relation=target_relation) %}

as someone did here https://getdbt.slack.com/archives/C2JRRQDTL/p1594844172456700?thread_ts=1594742649.422500&cid=C2JRRQDTL, but concerned about future code divergence/our team is very new to dbt so getting this out of the box would be great.

Additional context

@drewbanin suggested making an issue here https://getdbt.slack.com/archives/C2JRRQDTL/p1594820882440000?thread_ts=1594742649.422500&cid=C2JRRQDTL but I didn’t see one, so figured I’d make my first contribution! Seems like a straightforward override flag in the config, like expand_target_column_types=false would suffice?

Who will this benefit?

Anyone who would like the types in their tables to be immutable/fully specified by code. For example, we have some values that we expect to be no more than 16 characters long; an unintended expansion of those fields to the max varchar length means we’d lose desired validation of those types.

Are you interested in contributing this feature?

I can’t right now, but not for lack of desire; can see myself contributing in the future 😃

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
jtcohen6commented, Oct 5, 2020

@tashay sweet! Let me know if you’d like any support in getting started 😃

1reaction
tashaycommented, Oct 5, 2020

I’m interested in working on this! @jtcohen6

Read more comments on GitHub >

github_iconTop Results From Across the Web

Optionally skip expand_target_column_types during ... - GitHub
Would like to be able to skip expand_target_column_types in an incremental materialization; we'd like the types in our tables to be immutable/ ...
Read more >
Incremental models - dbt Developer Hub
By using the first syntax, which is more universal, dbt can ensure that the columns will be templated into your incremental model materialization...
Read more >
DBT Incremental models | iomete docs
Incremental models are built as tables in your data lake. ... Like the other materializations built into dbt, incremental models are defined with...
Read more >
Understanding dbt Incremental Strategies part 2/2 - Medium
The materialized configuration of the model is set to 'incremental'. Let's see how to do it right now! Configuring materialized = 'incremental'.
Read more >
Creating an Incremental Materialization | ClickHouse Docs
This results in similar limitations for large results set as for the table model. To overcome these limitations for large sets, the plugin...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found