question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

dashboard_link parameter cannot be set when using SLURMCluster from dask_jobqueue

See original GitHub issue

I am trying to use the dashboard with SLURMCluster, but setting dashboard_link=‘127.0.0.1:8787’ does not do anything. My code is as follows:

from dask_jobqueue import SLURMCluster

cluster = SLURMCluster(cores=32,
                     memory="100GB",
#                      project='P48500028',
#                      queue='premium',
#                      interface='ib0',
                    dashboard_link = '127.0.0.1:8787',
                     walltime='00:30:00')

cluster.scale(2)  # Start 100 workers in 100 jobs that match the description above

from dask.distributed import Client
client = Client(cluster) 

In fact the output of cluster is just:

Client
Scheduler: tcp://128.2.204.115:32776 | Cluster Workers: 2 Cores: 64 Memory: 200.00 GB

Am I missing something here?

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:8 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
lestevecommented, Sep 9, 2019

Weird, can you make sure that bokeh is installed in your environment.

Here is a conda command that creates an environment with the latest versions of dask, distributed and dask-jobqueue:

conda create -n testenv -c conda-forge dask distributed dask-jobqueue -y 

Activating this environment:

conda activate testenv

I can then execute the snippet above without any problems:

from dask_jobqueue import SLURMCluster

cluster = SLURMCluster(cores=1, memory='1GB', dashboard_address=':12123')
print(cluster.dashboard_link)

Side-comment (not related to your error): unless you have a good reason to use python 3.5, I would advise to use python >= 3.6.

1reaction
lestevecommented, Sep 9, 2019

Great to hear that! I am going to close this issue.

Agreed the error could be better or maybe there could be a warning earlier if bokeh is not installed to say that the dashboard is not going to be available …

Feel free to open an issue in dask/distributed about this!

Read more comments on GitHub >

github_iconTop Results From Across the Web

dask_jobqueue.SLURMCluster - Dask-Jobqueue
Launch Dask on a SLURM cluster. Parameters. queuestr. Destination queue for each worker job. Passed to #SBATCH -p option. projectstr. Deprecated: use ......
Read more >
Setting up a DASK cluster using dask-jobqueue
We show here how to deploy a DASK cluster on compute nodes using the SLURM ... SLURMcluster is going to allow us to...
Read more >
dask-jobqueue does not start any worker on slurm cluster
When running squeue , no job appear. All the tests are passing. Using LocalCluster() does work on the login node. Those are the...
Read more >
Dask jobqueue on Levante - the DKRZ documentation!
Using Dask jobqueue you can launch Dask cluster/workers as a Slurm jobs. ... from dask_jobqueue import SLURMCluster from dask.distributed import Client. Set ......
Read more >
Dask-jobqueue Documentation - Release 0.7.1 'Dask ...
3.1 Installing You can install dask-jobqueue with pip, conda, ... 3.5 Configuration Dask-jobqueue should be configured for your cluster so that it knows...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found