Try out dask-jobqueue development version on different clusters
See original GitHub issueThere has been a non-trivial refactoring in #307 to use SpecCluster
from distributed
. It would be great if people could test this on their cluster to avoid unexpected problems when we released the next version of dask-jobqueue
(probably a matter of one week or two):
-
PBSCluster
: @guillaumeeb @andersy005. This has already been tested while #307 was being worked on I think. -
HTCondor
: @matyasselmeci @cloudronin @mivade -
LSFCluster
: tested by @mrocklin (on Summit I think). Not sure who else would be a good person. -
SLURMCluster
: tested by @mrocklin. Not sure who else would be a good person. -
SGECluster
: I can test this. Maybe @ericmjl can try it out as well. -
OARCluster
: I can test this.
To install the dask-jobqueue
development version:
pip install `git+https://github.com/dask/dask-jobqueue`
You want to make sure that you have distributed >= 2.5.1
(see #342 for more details).
There is a list of known breaking changes: https://github.com/dask/dask-jobqueue/blob/master/docs/source/changelog.rst#changelog
If you spot some more breaking changes, let us know, so we can improve the changelog!
Issue Analytics
- State:
- Created 4 years ago
- Comments:23 (17 by maintainers)
Top Results From Across the Web
Configure Dask-Jobqueue
Many HPC systems have a variety of different queues to which you can submit jobs. These typically have names like “regular”, “debug”, and...
Read more >Advanced tips and tricks - Dask-Jobqueue
The universe of HPC clusters is extremely diverse, with different job schedulers, ... but is considered deprecated and will be removed in a...
Read more >dask_jobqueue.LSFCluster - Dask-Jobqueue
Launch Dask on a LSF cluster. Parameters. queuestr. Destination queue for each worker job. Passed to #BSUB -q option. projectstr.
Read more >dask_jobqueue.PBSCluster - Dask-Jobqueue
Launch Dask on a PBS cluster. Parameters. queuestr. Destination queue for each worker job. Passed to #PBS -q option. projectstr.
Read more >Dask-Jobqueue — Dask-jobqueue 0.8.1+2.gbee0e0c.dirty ...
The Dask-jobqueue project makes it easy to deploy Dask on common job queuing systems typically found ... academic research institutions, and other clusters....
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
OK, things here seem to have quieted down. If no one has any strong objections I plan to release this tomorrow.
0.7 is up on PyPI. Closing.