Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

DbtShellTask does not work without profiles_dir parameter

See original GitHub issue

Hi, profiles_dir parameter is marked as optional, but without that parameter DbtShellTask fails: TypeError('expected str, bytes or os.PathLike object, not NoneType')

It happens because of this line: https://github.com/PrefectHQ/prefect/blob/6ef1f36362a46762985dfcab27a3c4ea0ec8084a/src/prefect/tasks/dbt/dbt.py#L146

Issue Analytics

State:
Created 3 years ago
Reactions:3
Comments:5

Top GitHub Comments

2reactions

fertekcommented, Jul 28, 2020

dbt docs at https://docs.getdbt.com/dbt-cli/configure-your-profile/#using-a-custom-profile-directory says:

By default, dbt expects your profiles.yml file to be located in the ~/.dbt/ directory.

Therefore, I think that without profiles_dir parameter specified the ~/.dbt/ directory should be used.

1reaction

mhmcdonaldcommented, Jul 28, 2020

@fertek @Zaubeerer It’s been a while since I wrote this but I believe I left it as optional because you might have the location already set through the dbt envar DBT_PROFILES_DIR. If it wasn’t possible to set the dir this way, I would have made it required.

I think defaulting the profiles_dir path to ~/.dbt/ is a great idea, just as long as the DBT_PROFILES_DIR envar gets priority in determining the path location. This would be in keeping with how dbt checks for the profiles.yml file.