Support running without kubernetes, just docker

Proposed change

There’s nothing inherent about binderhub that requires kubernetes. What it needs really is:

1. A way to spawn repo2docker
2. A way to tell a JupyterHub to run users with a specific image

This is doable without kubernetes! If you look at where we use kubernetes, it is mostly only in build.py. We already support another build backend - FakeBuild.

I’d like to support an additional backend that lets you run binderhub on a single host, with docker + a jupyterhub with dockerspawner. This will make it much easier for people to set up ‘small’ binderhubs internally, and also exercise our abstraction for future users (who might want it on HPCs, for example).

Alternative options

Decide that we have a hard dependency on kubernetes, and other setups will not be supported. I don’t think we should do this though.

Who would use this feature?

1. People who want to run binderhub on a single node, since they don’t expect a lot of traffic (or it is authenticated)
2. People who want to code for different backends eventually. This would also involve finishing up @manics’s work in https://github.com/jupyterhub/repo2docker/pull/848 in repo2docker, but that can happen independent of work here.

(Optional): Suggest a solution

1. Turn Build in https://github.com/jupyterhub/binderhub/blob/00bd8304e99e9a4edbdfcca6fd98293fb1fb2e6a/binderhub/build.py#L18 into an abstract, well defined interface. I think this involves definint __init__, submit and stream_logs
2. Implement a DockerBuild that implements Build on top of the docker API
3. Investigate features we need that are deployed as part of the helm chart, rather than in the python package. Refactor as needed (keeping in mind needs of simplicity for mybinder.org)
4. Move our kubernetes imports to a file, and make it conditional. This helps us make kubernetes an optional dependency!
5. Add tests to make sure things work with just docker!

[yuvipanda]

Counterpoint to myself: if we can run BinderHub with just docker, it will make local testing of most of the logic way easier, maybe improving maintenance of the whole thing.

+1

We can also put the docker logic in a separate package, so we end up in a situation like we have with spawners. I use SimpleLocalProcessSpawner for testing, and that’s so much nicer than requiring k8s for it.

[manics]

👍 to putting the Docker logic in a separate package. I think just the act of defining interfaces (e.g. BinderBuilder and BinderLauncher?) will improve maintainability of the code through clear documentation, at the moment there’s an assumption you know a lot of Kubernetes concepts which can make the code hard to understand.