Support temp tables in Snowflake "table" materializations
See original GitHub issueDescribe the feature
Please allow us to specify a “temporary” option in the config that will create a temp table materialization.
Describe alternatives you’ve considered
I could create a custom materialization that does what is described, but the change seems sufficiently small and useful to go in DBT’s materialization.
Additional context
Snowflake offers two alternatives to permanent tables- temporary tables and transient tables. Temp tables are dropped at the end of the session while transient tables must be explicitly dropped, otherwise they incur charges. DBT’s table
materialization currently forces users to use transient tables, and this should be opt-out.
Who will this benefit?
Users that need temp tables for their projects, but don’t want those tables to persist between runs, would benefit from having those tables be temporary. Projects that have frequent logic changes to their intermediate transforms as well as large datasets fit this description.
Are you interested in contributing this feature?
I would be willing to make this change
Issue Analytics
- State:
- Created 3 years ago
- Reactions:3
- Comments:10 (4 by maintainers)
Yes, that’s still true. This rough mapping holds: one model = one materialization = one Snowflake session/connection (possibly more if additional queries are run before/after)
Hi @jtcohen6, we at Zocdoc have started looking into supporting concurrent DBT runs without threading (as separate processes). I wanted to double-check that your point above is still accurate- that Snowflake DBT opens and closes sessions between models. Is that still true?