question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Support temp tables in Snowflake "table" materializations

See original GitHub issue

Describe the feature

Please allow us to specify a “temporary” option in the config that will create a temp table materialization.

https://github.com/fishtown-analytics/dbt/blob/dev/marian-anderson/plugins/snowflake/dbt/include/snowflake/macros/materializations/table.sql#L26

Describe alternatives you’ve considered

I could create a custom materialization that does what is described, but the change seems sufficiently small and useful to go in DBT’s materialization.

Additional context

Snowflake offers two alternatives to permanent tables- temporary tables and transient tables. Temp tables are dropped at the end of the session while transient tables must be explicitly dropped, otherwise they incur charges. DBT’s table materialization currently forces users to use transient tables, and this should be opt-out.

Who will this benefit?

Users that need temp tables for their projects, but don’t want those tables to persist between runs, would benefit from having those tables be temporary. Projects that have frequent logic changes to their intermediate transforms as well as large datasets fit this description.

Are you interested in contributing this feature?

I would be willing to make this change

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:3
  • Comments:10 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
jtcohen6commented, Mar 29, 2022

Yes, that’s still true. This rough mapping holds: one model = one materialization = one Snowflake session/connection (possibly more if additional queries are run before/after)

1reaction
gil-walzer-zocdoccommented, Mar 28, 2022

Hi @jtcohen6, we at Zocdoc have started looking into supporting concurrent DBT runs without threading (as separate processes). I wanted to double-check that your point above is still accurate- that Snowflake DBT opens and closes sessions between models. Is that still true?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Working with Temporary and Transient Tables
In addition to permanent tables, which is the default table type when creating tables, Snowflake supports defining tables as either temporary or transient....
Read more >
Creating Temporary tables in DBT - Help
I am using snowflake and I need to create a temporary table from dbt and inserting that table using another table and using...
Read more >
Snowflake Temporary Table Made Easy: How to Create & Use?
In this article, we will look at how to create a Snowflake Temporary Table, the syntax, usage, and limitations, as well as some...
Read more >
What's the difference between using WITH clause and using ...
WITH clause in the complex query will require materialization which needs memory to store intermediate result . If you create TEMP table to...
Read more >
will DBT support temp table creation like create table #temp1 ...
You're right in thinking that dbt does not support temporary tables. That's because temporary tables only persist in a single session, ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found