max_clusters

Hello, @jcrist.

Unless I’m missing something (which is very possible), or it was renamed, it looks like the max_clusters option was removed during the re-architecture. There’s no reference to it in the latest docs or source.

Are there any plans to reintroduce it?

[jcrist]

There’s some tricky aspects to this, especially from the k8s side. I’m hoping to have time to work on it sometime soon (rather busy with work right now), but no promises. I’d be happy if you wanted to give it a go, just know it’s much less straightforward than the per-cluster limits implementation.

[jcrist]

Please correct me if I’m wrong, but it seems like DG makes the assumption that the namespaces will be generated ahead of time. This will be difficult given that our platform is dynamic as opposed to bespoke and we anticipate hundreds of users.

This is correct. Since dask-gateway doesn’t maintain a users table (a cluster has a user as metadata, but there is no table of users maintained by dask-gateway), I don’t think user-management things like this belong in dask-gateway. We could add some hooks to make it work, but they might not support all things you’d like to do. There’s also no way for dask-gateway to know it can delete the namespace, so any cleanups you’d want would have to be external to dask-gateway.

If you’re using JupyterHub, I’d do this by implementing the add_user hook in the JupyterHub authenticator (https://github.com/jupyterhub/jupyterhub/blob/21eaf0dd9f76c9eed1757280b90e580218db9f58/jupyterhub/auth.py#L588). This gets called once for every new user (also called on hub-restart, so it would need to debounce), and could create a namespace or any other resources needed for that user. This is nice in that it happens once external to dask-gateway, so the namespace check wouldn’t need to happen when creating each cluster. There’s also a delete_user method to delete the namespace/other-stuff, so cleanup could happen all within JupyterHub.

To summarize, two options (as I see them):

• Implement this outside of dask-gateway with whatever user-management system you’re using/building. Create the namespace (and whatever else) when a new user is created.
• Add a pre-cluster-creation hook in dask-gateway that runs your custom code whenever a new cluster is created. This would need to debounce multiple executions, as we’d have no way to know if it already ran for a user or not. It also wouldn’t support deleting the namespaces for no-longer-existing users, that would have to be done external to dask-gateway.

I’d prefer you use the first one if possible (less stuff for dask-gateway to do), but the latter is also fine.