Allow "direct" Docker host config

docker.py often lags Go interface by quite some time. For example, there’s support for --gpus in the CLI but not docker.py. I propose that we allow DockerSpawner to bypass HostConfig, and just send parameters directly to the Docker daemon. There’s a hack solution to support gpus in RSDockerSpawner, which would be better implemented if DockerSpawner allowed host_config to be modified after the create_host_config.

I think the general docker.py takes on converting arguments is flawed, that is, HostConfig shouldn’t be mapping Python names to Go config names. There are 63 parameters to HostConfig. This is problematic for a number of reasons, mostly because we can’t look at the Go code to figure out the names for things. It would be cleaner to just pass the Go parameters directly.

I’m happy to submit a PR, but this is something that obviously needs discussion. My hack works for now.

[robnagler]

@samon11 You will need to create a subclass of DockerSpawner. This can be done in the config, and then you need to override the docker method.

[acelere]

OK. Figured it out. I ran the stock NVIDIA container interactively with: docker run --gpus all -it --rm nvcr.io/nvidia/tensorflow:22.03-tf2-py3 and got the following message back:

So I need a new GPU 😃

Anyway, I guess the hint here is to run the container and check the errors…because not all of the output is presented to the user on Jupyterhub.

Cheers.