question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Airflow 2.1.0: Snowflake connection not enabling snowflake "role"

See original GitHub issue

Apache Airflow version: 2.1.0

Kubernetes version (if you are using kubernetes) (use kubectl version):

Environment:

  • Cloud provider or hardware configuration: AWS EKS, AWS ECS
  • OS (e.g. from /etc/os-release): Debian GNU/Linux 10
  • Kernel (e.g. uname -a):
  • Install tools:
  • Others:

What happened: When creating a snowflake connector using a snowflake hook(using UI), not able to add snowflake roles in the extras. here is the screenshot of the connection which I am creating Screen Shot 2021-06-29 at 1 31 33 PM

but once I am done saving it I lose the role and if I edit my connection this is what I see in the UI

Screen Shot 2021-06-29 at 1 32 18 PM

so Extra gets populated with all the other placeholders. I can see that https://github.com/apache/airflow/blob/4b14d93f25f237b8cca17f57352c253f292a111b/airflow/providers/snowflake/hooks/snowflake.py#L89 you guys are returning some value to the widget but it is not having a snowflake “role” definition.

https://github.com/apache/airflow/blob/4b14d93f25f237b8cca17f57352c253f292a111b/airflow/providers/snowflake/hooks/snowflake.py#L105

I also updated the JSON object and created it as a nested JSON {{"role": "test"}} following the placeholder from the above function but still, the behaviour remains the same.

What you expected to happen:

"placeholders": {
                'extra': json.dumps(
                    {
                        "role": "snowflake role",
                        "authenticator": "snowflake oauth",
                        "private_key_file": "private key",
                        "session_parameters": "session parameters",
                    },
                    indent=1,
                ),
                'host': 'snowflake hostname',
                'schema': 'snowflake schema',
                'login': 'snowflake username',
                'password': 'snowflake password',
                'extra__snowflake__account': 'snowflake account name',
                'extra__snowflake__warehouse': 'snowflake warehouse name',
                'extra__snowflake__database': 'snowflake db name',
                'extra__snowflake__region': 'snowflake hosted region',
                'extra__snowflake__aws_access_key_id': 'aws access key id (S3ToSnowflakeOperator)',
                'extra__snowflake__aws_secret_access_key': 'aws secret access key (S3ToSnowflakeOperator)',
            }

If I follow above code I should be able to pass the snowflake role into the Extras. But this behaiour is missing

How to reproduce it:

Create a connection with connection type snowflake in airflow UI.

Anything else we need to know:

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:7 (5 by maintainers)

github_iconTop GitHub Comments

2reactions
spniemansburgcommented, Aug 19, 2021

Issue is closed, but wanted to note that the same issue is happening when trying to add a private key file (using “private_key_file” parameter) in the Extra field. I am using Apache Airflow 2.1.0. Should I file a new bug report?

Read more comments on GitHub >

github_iconTop Results From Across the Web

apache-airflow-providers-snowflake Documentation
This version is yanked, as it contained problems when interacting with common.sql provider. Please install a version released afterwards. Update snowflake hook ...
Read more >
How to connect Apache Airflow to Snowflake and schedule ...
Configure snowflake connection in Airflow​​ 2. We can run below commands after activating the python virtual enviroment.
Read more >
Using apache-airflow-providers-snowflake on ... - Stack Overflow
I am importing below library and its working fine for with airflow 2.0.2 and snowflake-connector-python==2.4.3 .
Read more >
apache-airflow-providers-snowflake - Python package - Snyk
Learn more about apache-airflow-providers-snowflake: package ... 2.1.0. Features. Adding: Snowflake Role in snowflake provider hook (#16735) ...
Read more >
Intro to Airflow for ETL With Snowflake - YouTube
ETL is one of the most common data engineering use cases, and it's one where Airflow really shines. In this webinar, we'll cover...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found