Add ability for all operators to interact with storages of AWS/GCP/AZURE

Description

Currently every operator is doing XToY. so if someone wrote MySQLToGCS it doesn’t help someone who needs MySQLToS3.

Use case / motivation

It would be great if when PR is raised people will need only to handle the X part and provide a list of the Y part. Something like people need to write XtoDataframe or XtoFile and there is build in integration in Airflow that can handle the FileToS3 FileToGCS etc…

So when user is PR MySQLToFile Airflow will utilise this and auto create MySQLToGCS and MySQLToS3.

The idea is to build infrastructure layer once that will be automated for all.

[jrderuiter]

Would be interesting to implement this using fsspec (https://filesystem-spec.readthedocs.io/en/latest/). Such an implementation would provide an interface to all filesystems supported by fsspec, which includes Azure Blob/GDL2, S3 and GCS. You also get others like FTP and SFTP for free too (https://filesystem-spec.readthedocs.io/en/latest/api.html#implementations).

[dinigo]

@turbaszek I see how this is related, and I face this question frequently. But I think #8059 depends on having a common FS interface like #3526 sugests.

Right now we have (I might miss some) the following file providers: S3, GCS, Azure, SFTP/SSH, FTP, Samba. If we want to allow all operations between them it means it’s N^2 operators. It’s 36 operators. And will be more as other cloud providers are added (Alibaba, DO, …). I’ll give a look at the PR @ashb , thanks.