Support spawning to different clusters

Proposed change

Right now, the kubernetes pod is spawned in the same cluster as the hub pod. It would be great if we can configure it to be spawned in other remote clusters. One hub can then spawn into different cloud regions, which is very helpful when dealing with cloud datasets.

The kubernetes API can easily be accessed remotely, but the hub and proxy pod need to find a way to send traffic to the user pod. We can find ways to tunnel this traffic through without much work. My favorite way is to use kubectl port-forward, also used by my earlier expeirments with accessing dask-kubernetes remotely and now dask-kubernetes itself.

Alternative options

1. Deploy one hub per cluster users want to spawn into. This is more complicated logistically, and for the user.
2. Make a Service object for each pod, and expose it to the internet via a LoadBalancer. This can receive traffic from the hub and proxy pod

Who would use this feature?

Anyone interested in accssing compute near datasets stored across multiple cloud providers or regions

(Optional): Suggest a solution

• Override get_pod_url to start a kubectl port-forward on a free port, to the pod IP on the remote cluster
• Make sure that c.JupyterHub.hub_connect_url is something that the pod can connect to. This could be over https on the public internet, or something else.
• Figure out how to specify which kubernetes cluster the API will need to connect to

[yuvipanda]

@nreith I actually ended up building a separate spawner for this, and it works fairly well - https://github.com/yuvipanda/jupyterhub-multicluster-kubespawner.

[TiPPeX2]

@yuvipanda , Thanks for your great work, I appreciate it very much!

Currently the KubeSpawner is only able to spawn on it’s own namespace(due to reflectors) Is the multicluster related to multi namespace by any means(or only clusters)?

I remember there is a configuration to give full cluster permissions to the hub allowing to create namespaces per user. But this is not the case.

I would like to have a single hub, which can spawn on multiple Kube namespaces(which are not the same as the hub) I have a FB of Kubespawner which changes how reflectors work, and added permission to each namespace I want into the Jupyterhub serviceAccount.

Was curios if in your sub-repo there is a way to implement above scenario, or if my implementation would have any use case for others so I could maybe open a PR and issue about it?.

We did it for multiple reasons:

1. Single place for all users(instead of having a Jupyterhub per namespace)
2. Minimal permission to the Jupyterhub, only have permissions on selected namespaces.
3. Reflectors are only looking on spawned namespaces for events, instead of the entire cluster which is quite big.

Thanks for your time!