Use default google cloud project if not supplied?

Describe the feature

Currently using BigQuery requires defining your project in profiles.yml: https://docs.getdbt.com/reference/warehouse-profiles/bigquery-profile/

Google Cloud APIs generally fall back to the default project when one isn’t specified. This is helpful for code that runs in multiple project environments — it’ll reference the datasets in whatever project it’s running in.

So the feature would be to align dbt with that standard, and allow for:

my-bigquery-db:
  target: dev
  outputs:
    dev:
      type: bigquery
      method: oauth
      # project: [GCP project id] <- uses the current project
      dataset: [the name of your dbt dataset] # You can also use "schema" here
      threads: [1 or more]
      timeout_seconds: 300
      location: US # Optional, one of US or EU
      priority: interactive
      retries: 1

Describe alternatives you’ve considered

Currently we have something like:

nimbus:
  target: main
  outputs:
    user: main:
      type: bigquery
      method: oauth
      project: "{{ env_var('PROJECT', 'project_foo') }}"

…and set $PROJECT to the result of gcloud config get-value project. This is OK, but some cruft.

(and if I’m missing something and there’s any easy solution to this, that would be gratefully received!)

Who will this benefit?

BigQuery users, particularly those running across dev and prod environments

Are you interested in contributing this feature?

Not right now, given my other OSS work, but would be keen to contribute to dbt at some point!

Thank you!

[jtcohen6]

Thanks for the detailed proposal, @max-sixty. I think it makes sense to fall back to the default project configured by the gcloud user / service account if it’s not specified in profiles.yml.

This isn’t a change we would prioritize, and I’m glad to see you have a workaround in the meantime. I imagine it could be quite straightforward. I’ll mark this a good first issue, for whenever you (or another community member) has the time.

[yu-iskw]

There are two approaches to set scopes to the oauth method, if there is no way to grant scopes to a google cloud account.

1. Pass the SCOPE constant to google.auth.default for the oauth method too.
2. Add a sub map to pass scopes to profiles.yml like below.

my-bigquery-db:
  target: dev
  outputs:
    dev:
      type: bigquery
      method: oauth
      project: [GCP project id]
      dataset: [the name of your dbt dataset] # You can also use "schema" here
      threads: [1 or more]
      timeout_seconds: 300
      location: US # Optional, one of US or EU
      priority: interactive
      retries: 1
      scopes:
        - https://www.googleapis.com/auth/bigquery
        - https://www.googleapis.com/auth/cloud-platform
        - https://www.googleapis.com/auth/drive