Unknown connection types fail in cryptic ways

Apache Airflow version

2.2.4 (latest released)

What happened

I created a connection like:

airflow connections add fsconn --conn-host /tmp --conn-type File

When I really should have created it like:

airflow connections add fsconn --conn-host /tmp --conn-type fs

While using this connection, I found that FileSensor would only work if I provided absolute paths. Relative paths would cause the sensor to timeout because it couldn’t find the file. Using fs instead of File caused the FileSensor to start working like I expected.

What you think should happen instead

Ideally I’d have gotten an error when I tried to create the connection with an invalid type.

Or if that’s not practical, then I should have gotten an error in the task logs when I tried to use the FileSensor with a connection of the wrong type.

How to reproduce

No response

Operating System

debian (in docker)

Versions of Apache Airflow Providers

n/a

Deployment

Astronomer

Deployment details

astro dev start

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project’s Code of Conduct

[potiuk]

I think we shoudl simply check - whenever connection is added/updated whether the type is “Valid” and print a warning in the CLI. As @eladkal suggested, it should set the type of the connection to generic type if the connection type is not valid.

And Valid means two things:

• it is on the list of built-in connection tupes
• you can query the Providers Manager to see which connection types are present via providers. It has “hooks” property that has list of HookInfo objects which contains connection_type

The warning likely should print the list of valid connection types.

This is important to use ProvidersManager for that, because it automatically discovers which providers are installed, so it will prevent (or rather warn) if you try to create a connection for a provider that you do not have installed.

[Bowrna]

Shall I fix this issue?