Support for Dask Gateway clusters from config
See original GitHub issueRight now, IIUC, to create a cluster using the button the config takes a python class, args, and kwargs to create the cluster. This isn’t flexible enough for dask-gateway, which requires creating an intermediate Gateway object.
Two options
- Expand the logic of the lab extension’s cluster creation to take a snippet of code to run.
- Update
dask-gateway
to have a “simple” way of creating a cluster that just uses the defaults (cc @jcrist).
https://github.com/dask/dask-gateway/issues/55 is related, but more focused on expanding dask-labextension to take advantage of dask-gateway
Issue Analytics
- State:
- Created 3 years ago
- Reactions:1
- Comments:8 (5 by maintainers)
Top Results From Across the Web
Exposing Cluster Options - Dask Gateway
By default cluster configuration (e.g. worker memory, docker image, etc…) ... This supports dynamic configuration options that depend on the requesting user.
Read more >Dask-gateway: Cluster Options on K8s Setup empty
dask -gateway version is 0.9.0. EDIT: I should add that everything else works like a charm. EDIT2: I get a weird 401 on...
Read more >Using Dask Gateway — Pangeo Gallery documentation
We've done most of the configuration for you. All that remains for you is to either create a new cluster or connect to...
Read more >Deploy Dask Gateway with JupyterHub on Kubernetes
Preparation · Launch dask gateway · Modify the JupyterHub configuration · Create a dask cluster.
Read more >Dask Gateway, set worker resources - Stack Overflow
How can this be done, either when the cluster is created using the config.yaml of the helm chart (ideally, a field in the...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
dask-gateway doesn’t require creating an intermediate
Gateway
object - you can already calldask_gateway.GatewayCluster
directly (we might want to update our docs to better show this). I’ve verified things work with dask-labextension with default parameters just like any other cluster.I see that @consideRatio found a fix using the
GatewayCluster
-class (https://github.com/dask/dask-labextension/issues/203). Confirmed to work with the daskhub helm-chart:However, the
DASK DASHBOARD URL
is not set correctly as mentioned by Erik in the issue above.