[CT-1358] [Feature] Add column type constraints as dbt native configs

Is this your first time submitting a feature request?

• I have read the expectations for open source contributors
• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

Problem:

• Data types are created after the fact(assumed by the database) and/or a dbt user has to explicitly cast data types in their SQL. Because of this, there isn’t much incentive to assure dbt models have expected data types for consumers to depend on(it’s tedious work 😕). This may not be a problem for 50 dbt models, but becomes a big problem when you have 2000+ dbt models. There should be an easier way to configure data type constraints for any dbt models and provide options to enable/disable data type enforcement(think: what mypy does for the python static typing experience).

Solution:

• Bring data types as native to the dbt experience
• Example BigQuery Constraints Macro:https://gist.github.com/sungchun12/f7ea081773ae824a83294649530d6e41
• Related discussion: https://github.com/dbt-labs/dbt-core/discussions/5244#discussioncomment-3880254

Describe alternatives you’ve considered

Custom Materialization Macro in BigQuery: https://www.loom.com/share/1f1f190e66254d12984962c613e8082d

This is a general design pattern, but uses meta configs to enable this functionality. This is brittle because it injects custom configs into the manifest that aren’t guaranteed to behave in expected ways.

Who will this benefit?

dbt users with large projects that need extra robustness in their developer experience and downstream users get the data they expect: data types and all.

Are you interested in contributing this feature?

yes, and I’ll be working with someone in the community @jonathanneo

Anything else?

Research:

• Postgres Constraints: https://www.postgresql.org/docs/current/ddl-constraints.html#id-1.5.4.6.6
• Snowflake ONLY enforces not null constraints, the rest are nice to have metadata: https://docs.snowflake.com/en/sql-reference/constraints-overview.html#supported-constraint-types
• BigQuery Macro Demo: https://www.loom.com/share/1f1f190e66254d12984962c613e8082d

Considerations:

• have the configs embedded in dbt-core and then each adapter can take stable configs to adjust table materialization macros
• Enforce column positions based on schema config
• dbt-core contains the hub of native configs while adapters own specific implementations as each database has nuances to which enforceable constraints are valid.
• think about not null and default values for a table
• Work with Jon Neo from Canva
• Include a check constraint as pgsql enables it?
• What if constraints could be defined once within a subfolder path in dbt_project.yml and then in automatically creates not null constraints in the DDL?
• Leverage how seed configs enable data types: https://docs.getdbt.com/reference/resource-configs/column_types

[joshuataylor]

I volunteer to help contribute the Snowflake portions and help test on a real world project. This will really help with incremental updates especially!

[jonathanneo]

Good spike! Just in time too, more and more data warehouse providers are adding constraints into their table definitions. For example, databricks have just added table constraints in October 2022: https://docs.databricks.com/tables/constraints.html