Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Add support for impersonation in GKEStartPodOperator, DataflowCreateJavaJobOperator and DataflowCreatePythonJobOperator

See original GitHub issue

Description

GKEStartPodOperator, DataflowCreateJavaJobOperator and DataflowCreatePythonJobOperator do not support direct impersonation of a service account in Google services.

Use case / motivation

GKEStartPodOperator, DataflowCreateJavaJobOperator and DataflowCreatePythonJobOperator, in contrary to other Google operators, do not use Credentials class for authentication, so they require individual approach to support direct impersonation in them.

In case of GKEStartPodOperator it seems it should suffice to add --impersonate-service-account to the gcloud container clusters get-credentials command. However, this way we will not be able to use chain of service accounts, like in the rest of Google operators.

In case of DataflowCreateJavaJobOperator and DataflowCreatePythonJobOperator, some changes in provide_gcp_credential_file_as_context will probably be needed.

Issue Analytics

State:
Created 3 years ago
Comments:7 (7 by maintainers)

Top GitHub Comments

1reaction

mik-lajcommented, Oct 5, 2020

@rajatsri28 Awesome! I assigned you to this ticket. 🐈

1reaction

rajatsri28commented, Oct 5, 2020

Hi @olchas , Can I pick this up?

Top Results From Across the Web

airflow.providers.google.cloud.operators.dataflow

Optional service account to impersonate using short-term credentials, or chained list of accounts required to get the access_token of the last account in ......

pip install apache-airflow-providers-google==1.0.0b2 - PyPI

You can install this package on top of an existing airflow 2. ... 5eacc1642, 2020-07-22, Add support for impersonation in GCP hooks (#9915)....

GKEStartPodOperator - Astronomer Registry

Install the Google provider package into your Airflow environment. ... impersonation_chainOptional service account to impersonate using ...

RELEASE_NOTES.rst - airflow - Git at Google

Adding support for owner links in the Dags view UI (#25280) ... Direct impersonation added to operators communicating with Google services.

Package apache-airflow-backport-providers-google - Gitee

DataflowCreateJavaJobOperator · contrib.operators.dataflow_operator. ... Raise exception when GCP credential doesn't support account impersonation (#8213).