Feature Request: Add plugin support to enable new protocols

Problem description

I was thinking maybe we could provide a plugin system so that people can build their own packages and provide support for other file systems/protocols, (e.g. dropbox:// to access a file on Dropbox) without having to put all of them in smart_open() and make it a heavyweight package. For example, one could implement https://github.com/RaRe-Technologies/smart_open/issues/228 as a plugin so that we don’t need to make everyone install the Azure Python client even if they’re never going to use it.

There is prior art around this. setuptools supports entry points for building plugins, and we could use pluggy to provide hooks for recognizing and implementing file system protocols. PyFilesystem2 does something very similar already so we could look there for inspiration.

[mpenkov]

It’s an interesting idea, but I don’t think it’s worth the effort at the moment, because it introduces additional complexity and doesn’t solve any problems that are hurting us at the moment.

For example, if we started to get overwhelmed with dependencies like boto3, gcs, etc. (currently, it’s manageable), then we could make them optional, like we do with paramiko.

If people were contributing backends faster than we could merge PRs (again, we’re nowhere near that now) then it would make sense to think about plugins.

[dargueta]

Will do, but don’t count on it being anytime soon.