Colab InteractiveContext Unable to Determine ProjectID for BQ

When trying to use TFX in Colab, specifically the BigQueryExampleGen, there is an issue where we get the following error regarding the GCP project_id: “Project was not passed and could not be determined from the environment.”

gcloud config list shows the project is recognized, so the issue is within the last, or second-to-last line.

Perhaps there is a parameter within InteractiveContext() or BigQueryExampleGen(), but I couldn’t find any documentation on this. Is there a way to specify the project_id in any of these functions? Or is this a bug?

Below is the exact replica of the issue:

from tfx.orchestration.experimental.interactive.interactive_context import InteractiveContext
from tfx.components.example_gen.big_query_example_gen.component import BigQueryExampleGen
from google.cloud import bigquery
from google.colab import auth

auth.authenticate_user( )
project_id = 'Project-ID'
!gcloud config set project {project_id}
!gcloud config list


context = InteractiveContext()

QUERY = '''
SELECT * 
FROM `bigquery-public-data.chicago_taxi_trips.taxi_trips` 
LIMIT 1000
'''
example_gen = BigQueryExampleGen(query=QUERY )
context.run(example_gen)

[charlesccychen]

@luischinchillagarcia can you try installing pip install -i https://test.pypi.org/simple/ tfx==0.16.0.dev20191105 and then adding context.run(..., beam_pipeline_args=['--project', 'my-project'])?

[1025KB]

#888 is merged into master, could you try again?