Defining vars, folder-level configs outside dbt_project.yml

Describe the feature

From @benjaminsingleton:

I’d like to use project level variables more, but I’m concerned about bloat to my already large dbt-project.yml file. I think it would be helpful if I could create a variables.yml file that could be imported in dbt-project.yml . And for that matter, the same could be done for other configurations in the dbt_project.yml file. I think having the ability to separate configurations into different files might make for improved modularity / separation of concerns (particularly for large projects), not to mention fewer merge conflicts. CC @jrandrews

Describe alternatives you’ve considered

• We’re already thinking of enabling some configs in resource-YAML files (#2401), but these would be at the level of the individual resource (model/seed/snapshot/etc) only
• The dbt_project.yml gets really really big??

Additional context

• I don’t think this has any correspondence to v1.0. It’s a nice thing to have, and we can could do it before, after, any time without it being a breaking change in any way.

Who will this benefit?

• Developers and maintainers of increasingly big dbt projects

[krazavet-tinyclues]

Hello 👋

In our company, we are using a lot DBT in a multi tenant context. For that purpose, we rely a lot on DBT variables with which we propagate the client configuration. Those configurations could be really different from a client to another. Sometimes we have faced the following issue argument list too long: dbt, which is due to the large config payload (e.g. some of them could reach more than 600Kb).

We did not find a proper workaround for now. Passing a file path instead of a payload for our variables would probably solve our issue. This is why we are keen to know if there is any chance you are going to consider such feature for DBT ? (cc. @jtcohen6)

Thank you in advance 🙏

[itechprasanth]

I agree. Hope this will get implemented soon as it is always a good practice to modularize the configurations, rather than having everything in same single file.