404 for cluster dashboard

I setup dask-gateway on a microk8s single-node cluster for testing. My config is minimal, only setting the gateway.backend.image to my own custom image. I separately run jupyterhub, with no authentication.

From a notebook, I can create a cluster and run computations, but: I cannot see the dashboards.

With the cluster object, I get a the following URL: http://10.64.140.43/clusters/starmap.06630af66e734c3c9463348e2dc36344/status

With the client object, I get: http://10.64.140.43:8787/status

Going to either the cluster url gives a 404. Going to the client url times out.

The IP address (also the one I give to the GatewayCluster c’tor) is given by the metallb load balancer, and shows up as the external-ip for traefik-dask-gateway:

NAMESPACE            NAME                                                    TYPE           CLUSTER-IP       EXTERNAL-IP    PORT(S)                      AGE
starmap              service/traefik-dask-gateway                            LoadBalancer   10.152.183.38    10.64.140.43   80:31625/TCP                 79m

Did I miss a bit of config, either for microk8s or dask-gateway?

[TomAugspurger]

Well that was one of the more frustrating debugging sessions 😃

diff --git a/distributed/scheduler.py b/distributed/scheduler.py
index 33bd6373..2887abec 100644
--- a/distributed/scheduler.py
+++ b/distributed/scheduler.py
@@ -1133,7 +1133,7 @@ class Scheduler(ServerNode):
         )
         self.start_http_server(routes, dashboard_address, default_port=8787)
 
-        if dashboard:
+        if dashboard or dashboard_address:
             try:
                 import distributed.dashboard.scheduler
             except ImportError:

Not sure why I didn’t think to compare the args of dask-scheduler and dask-gateway-scheduler earlier. Going to make sure that’s an appropriate fix for distributed, and will look into whether dask-gateway needs to change at all.

[TomAugspurger]

My apologies for the churn here Tom. Thank you for resolving my mess here.

No worries, It’s just a one line fix 😃

We’ll release distributed 2.15.2 later today with the fix, which will kick off builds on conda-forge, pangeo-notebook conda metapackages, and eventually the pangeo docker images.

So the bad versions will be

• distributed 2.15.0 and 2.15.1
• pangeo-dask 2020.04.25, 2020.04.28
• pangeo-notebook 2020.04.28, 2020.04.28, 2020.04.29
• pangeo/base-notebook 2020.04.27, 2020.04.28, 1391655