Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Client Startup fails when `n_workers` more than physical cores
See original GitHub issueIf I try to start a local cluster with either
from distributed import Client
client = Client()
or
from distributed import Client, LocalCluster
cluster = LocalCluster()
client = Client(cluster)
I get the first error message attached below.
If I start up a cluster with a number of workers equal to the number of physical cores (in my case 24) like this:
from distributed import Client, LocalCluster
cluster = LocalCluster(n_workers=24)
client = Client(cluster)
The I get the second error message below.
If I instead split the cluster and client creation across two ipython lines:
In [1]: from distributed import Client, LocalCluster
...: cluster = LocalCluster(n_workers=24)
...:
In [2]: client = Client(cluster)
Then everything works fine.
Version numbers
Python 3.6.4 | packaged by conda-forge | (default, Dec 24 2017, 10:11:43) [MSC v.1900 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import distributed
In [2]: distributed.__version__
Out[2]: '1.21.4'
First error message
distributed.diskutils - WARNING - Found stale lock file and directory 'C:\\Users\\hoffmand\\dask-worker-space\\worker-3cuu_1r2', purging
distributed.nanny - WARNING - Worker process 34896 was killed by signal 15
distributed.nanny - WARNING - Worker process 32476 was killed by signal 15
distributed.nanny - WARNING - Worker process 38328 was killed by signal 15
distributed.nanny - WARNING - Worker process 4656 was killed by signal 15
distributed.nanny - WARNING - Worker process 27724 was killed by signal 15
distributed.nanny - WARNING - Worker process 35980 was killed by signal 15
distributed.nanny - WARNING - Worker process 36492 was killed by signal 15
distributed.nanny - WARNING - Worker process 38604 was killed by signal 15
distributed.nanny - WARNING - Worker process 31412 was killed by signal 15
distributed.nanny - WARNING - Worker process 21784 was killed by signal 15
distributed.nanny - WARNING - Worker process 22328 was killed by signal 15
distributed.nanny - WARNING - Worker process 34176 was killed by signal 15
distributed.nanny - WARNING - Worker process 29900 was killed by signal 15
distributed.nanny - WARNING - Worker process 15960 was killed by signal 15
distributed.nanny - WARNING - Worker process 27976 was killed by signal 15
distributed.nanny - WARNING - Worker process 31276 was killed by signal 15
distributed.nanny - WARNING - Worker process 38372 was killed by signal 15
distributed.nanny - WARNING - Worker process 41452 was killed by signal 15
distributed.nanny - WARNING - Worker process 25404 was killed by signal 15
distributed.nanny - WARNING - Worker process 22332 was killed by signal 15
distributed.nanny - WARNING - Worker process 27848 was killed by signal 15
distributed.nanny - WARNING - Worker process 16568 was killed by signal 15
distributed.nanny - WARNING - Worker process 35852 was killed by signal 15
distributed.nanny - WARNING - Worker process 22480 was killed by signal 15
distributed.nanny - WARNING - Worker process 42912 was killed by signal 15
distributed.nanny - WARNING - Worker process 30000 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 35856 was killed by signal 15
distributed.nanny - WARNING - Worker process 20356 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 14652 was killed by signal 15
distributed.nanny - WARNING - Worker process 38236 was killed by signal 15
distributed.nanny - WARNING - Worker process 3508 was killed by signal 15
distributed.nanny - WARNING - Worker process 24536 was killed by signal 15
distributed.nanny - WARNING - Worker process 40936 was killed by signal 15
distributed.nanny - WARNING - Worker process 15712 was killed by signal 15
distributed.nanny - WARNING - Worker process 20392 was killed by signal 15
distributed.nanny - WARNING - Worker process 29108 was killed by signal 15
distributed.nanny - WARNING - Worker process 20636 was killed by signal 15
distributed.nanny - WARNING - Worker process 17928 was killed by signal 15
distributed.nanny - WARNING - Worker process 11680 was killed by signal 15
distributed.nanny - WARNING - Worker process 41256 was killed by signal 15
distributed.nanny - WARNING - Worker process 5512 was killed by signal 15
distributed.nanny - WARNING - Worker process 33636 was killed by signal 15
distributed.nanny - WARNING - Worker process 28364 was killed by signal 15
distributed.nanny - WARNING - Worker process 26468 was killed by signal 15
distributed.nanny - WARNING - Worker process 1876 was killed by signal 15
distributed.nanny - WARNING - Worker process 29364 was killed by signal 15
distributed.nanny - WARNING - Worker process 22116 was killed by signal 15
distributed.nanny - WARNING - Worker process 11620 was killed by signal 15
distributed.nanny - WARNING - Worker process 13600 was killed by signal 15
distributed.nanny - WARNING - Worker process 11772 was killed by signal 15
distributed.nanny - WARNING - Worker process 21972 was killed by signal 15
distributed.nanny - WARNING - Worker process 12972 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 32868 was killed by signal 15
distributed.nanny - WARNING - Worker process 28140 was killed by signal 15
distributed.nanny - WARNING - Worker process 29204 was killed by signal 15
distributed.nanny - WARNING - Worker process 19396 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 28128 was killed by signal 15
distributed.nanny - WARNING - Worker process 41112 was killed by signal 15
distributed.nanny - WARNING - Worker process 26668 was killed by signal 15
distributed.nanny - WARNING - Worker process 35856 was killed by signal 15
distributed.nanny - WARNING - Worker process 4796 was killed by signal 15
distributed.nanny - WARNING - Worker process 21548 was killed by signal 15
distributed.nanny - WARNING - Worker process 13384 was killed by signal 15
distributed.nanny - WARNING - Worker process 4388 was killed by signal 15
distributed.nanny - WARNING - Worker process 20356 was killed by signal 15
distributed.nanny - WARNING - Worker process 29912 was killed by signal 15
distributed.nanny - WARNING - Worker process 34136 was killed by signal 15
distributed.nanny - WARNING - Worker process 15276 was killed by signal 15
distributed.nanny - WARNING - Worker process 14264 was killed by signal 15
distributed.nanny - WARNING - Worker process 14988 was killed by signal 15
distributed.nanny - WARNING - Worker process 38148 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 29820 was killed by signal 15
distributed.nanny - WARNING - Worker process 28832 was killed by signal 15
distributed.nanny - WARNING - Worker process 42856 was killed by signal 15
distributed.nanny - WARNING - Worker process 7164 was killed by signal 15
distributed.nanny - WARNING - Worker process 40672 was killed by signal 15
distributed.nanny - WARNING - Worker process 38848 was killed by signal 15
distributed.nanny - WARNING - Worker process 13964 was killed by signal 15
distributed.nanny - WARNING - Worker process 11836 was killed by signal 15
distributed.nanny - WARNING - Worker process 25336 was killed by signal 15
distributed.nanny - WARNING - Worker process 18016 was killed by signal 15
distributed.nanny - WARNING - Worker process 30752 was killed by signal 15
distributed.nanny - WARNING - Worker process 5888 was killed by signal 15
distributed.nanny - WARNING - Worker process 31384 was killed by signal 15
distributed.nanny - WARNING - Worker process 24308 was killed by signal 15
distributed.nanny - WARNING - Worker process 33556 was killed by signal 15
distributed.nanny - WARNING - Worker process 30004 was killed by signal 15
distributed.nanny - WARNING - Worker process 35196 was killed by signal 15
distributed.nanny - WARNING - Worker process 32328 was killed by signal 15
distributed.nanny - WARNING - Worker process 40244 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 27192 was killed by signal 15
distributed.nanny - WARNING - Worker process 2088 was killed by signal 15
distributed.nanny - WARNING - Worker process 27796 was killed by signal 15
distributed.nanny - WARNING - Worker process 37980 was killed by signal 15
distributed.nanny - WARNING - Worker process 16060 was killed by signal 15
distributed.nanny - WARNING - Worker process 4164 was killed by signal 15
distributed.nanny - WARNING - Worker process 23448 was killed by signal 15
distributed.nanny - WARNING - Worker process 37216 was killed by signal 15
distributed.nanny - WARNING - Worker process 11076 was killed by signal 15
distributed.nanny - WARNING - Worker process 28364 was killed by signal 15
distributed.nanny - WARNING - Worker process 17432 was killed by signal 15
distributed.nanny - WARNING - Worker process 41296 was killed by signal 15
distributed.nanny - WARNING - Worker process 27968 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 15748 was killed by signal 15
distributed.nanny - WARNING - Worker process 30300 was killed by signal 15
distributed.nanny - WARNING - Worker process 12972 was killed by signal 15
distributed.nanny - WARNING - Worker process 41256 was killed by signal 15
distributed.nanny - WARNING - Worker process 13600 was killed by signal 15
distributed.nanny - WARNING - Worker process 38316 was killed by signal 15
distributed.nanny - WARNING - Worker process 20636 was killed by signal 15
distributed.nanny - WARNING - Worker process 1876 was killed by signal 15
distributed.nanny - WARNING - Worker process 22116 was killed by signal 15
distributed.nanny - WARNING - Worker process 29108 was killed by signal 15
distributed.nanny - WARNING - Worker process 15712 was killed by signal 15
distributed.nanny - WARNING - Worker process 34096 was killed by signal 15
distributed.nanny - WARNING - Worker process 25096 was killed by signal 15
distributed.nanny - WARNING - Worker process 11620 was killed by signal 15
distributed.nanny - WARNING - Worker process 42664 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 20876 was killed by signal 15
distributed.nanny - WARNING - Worker process 41648 was killed by signal 15
distributed.nanny - WARNING - Worker process 34552 was killed by signal 15
distributed.nanny - WARNING - Worker process 33064 was killed by signal 15
distributed.nanny - WARNING - Worker process 14472 was killed by signal 15
distributed.nanny - WARNING - Worker process 14416 was killed by signal 15
distributed.nanny - WARNING - Worker process 32632 was killed by signal 15
distributed.nanny - WARNING - Worker process 17632 was killed by signal 15
distributed.nanny - WARNING - Worker process 16112 was killed by signal 15
distributed.nanny - WARNING - Worker process 37052 was killed by signal 15
distributed.nanny - WARNING - Worker process 24640 was killed by signal 15
distributed.nanny - WARNING - Worker process 27552 was killed by signal 15
distributed.nanny - WARNING - Worker process 13080 was killed by signal 15
distributed.nanny - WARNING - Worker process 43000 was killed by signal 15
distributed.nanny - WARNING - Worker process 23788 was killed by signal 15
distributed.nanny - WARNING - Worker process 22428 was killed by signal 15
distributed.nanny - WARNING - Worker process 32724 was killed by signal 15
distributed.nanny - WARNING - Worker process 23444 was killed by signal 15
distributed.nanny - WARNING - Worker process 31992 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 30448 was killed by signal 15
distributed.nanny - WARNING - Worker process 20820 was killed by signal 15
distributed.nanny - WARNING - Worker process 26496 was killed by signal 15
distributed.nanny - WARNING - Worker process 14736 was killed by signal 15
distributed.nanny - WARNING - Worker process 35228 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 39280 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 31440 was killed by signal 15
distributed.nanny - WARNING - Worker process 15280 was killed by signal 15
distributed.nanny - WARNING - Worker process 15928 was killed by signal 15
distributed.nanny - WARNING - Worker process 41608 was killed by signal 15
distributed.nanny - WARNING - Worker process 15580 was killed by signal 15
distributed.nanny - WARNING - Worker process 37900 was killed by signal 15
distributed.nanny - WARNING - Worker process 37044 was killed by signal 15
distributed.nanny - WARNING - Worker process 39156 was killed by signal 15
distributed.nanny - WARNING - Worker process 38660 was killed by signal 15
distributed.nanny - WARNING - Worker process 33648 was killed by signal 15
distributed.nanny - WARNING - Worker process 40636 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 19972 was killed by signal 15
distributed.nanny - WARNING - Worker process 18264 was killed by signal 15
distributed.nanny - WARNING - Worker process 42500 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 21276 was killed by signal 15
distributed.nanny - WARNING - Worker process 33968 was killed by signal 15
distributed.nanny - WARNING - Worker process 17936 was killed by signal 15
distributed.nanny - WARNING - Worker process 20764 was killed by signal 15
distributed.nanny - WARNING - Worker process 26232 was killed by signal 15
distributed.nanny - WARNING - Worker process 444 was killed by signal 15
distributed.nanny - WARNING - Worker process 10248 was killed by signal 15
distributed.nanny - WARNING - Worker process 14888 was killed by signal 15
distributed.nanny - WARNING - Worker process 33660 was killed by signal 15
distributed.nanny - WARNING - Worker process 15304 was killed by signal 15
distributed.nanny - WARNING - Worker process 12712 was killed by signal 15
distributed.nanny - WARNING - Worker process 37832 was killed by signal 15
distributed.nanny - WARNING - Worker process 20908 was killed by signal 15
distributed.nanny - WARNING - Worker process 40840 was killed by signal 15
distributed.nanny - WARNING - Worker process 21160 was killed by signal 15
distributed.nanny - WARNING - Worker process 41156 was killed by signal 15
distributed.nanny - WARNING - Worker process 36012 was killed by signal 15
distributed.nanny - WARNING - Worker process 41540 was killed by signal 15
distributed.nanny - WARNING - Worker process 36768 was killed by signal 15
distributed.nanny - WARNING - Worker process 42028 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 13856 was killed by signal 15
distributed.nanny - WARNING - Worker process 37944 was killed by signal 15
distributed.nanny - WARNING - Worker process 24736 was killed by signal 15
distributed.nanny - WARNING - Worker process 9760 was killed by signal 15
distributed.nanny - WARNING - Worker process 492 was killed by signal 15
distributed.nanny - WARNING - Worker process 15940 was killed by signal 15
distributed.nanny - WARNING - Worker process 13536 was killed by signal 15
distributed.nanny - WARNING - Worker process 12696 was killed by signal 15
distributed.nanny - WARNING - Worker process 29692 was killed by signal 15
distributed.nanny - WARNING - Worker process 22316 was killed by signal 15
distributed.nanny - WARNING - Worker process 5376 was killed by signal 15
distributed.nanny - WARNING - Worker process 25132 was killed by signal 15
distributed.nanny - WARNING - Worker process 25996 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 34720 was killed by signal 15
distributed.nanny - WARNING - Worker process 38036 was killed by signal 15
distributed.nanny - WARNING - Worker process 26032 was killed by signal 15
distributed.nanny - WARNING - Worker process 19060 was killed by signal 15
distributed.nanny - WARNING - Worker process 39812 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 24728 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 33268 was killed by signal 15
distributed.nanny - WARNING - Worker process 34176 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 3044 was killed by signal 15
distributed.nanny - WARNING - Worker process 14140 was killed by signal 15
distributed.nanny - WARNING - Worker process 13736 was killed by signal 15
distributed.nanny - WARNING - Worker process 22408 was killed by signal 15
distributed.nanny - WARNING - Worker process 7108 was killed by signal 15
distributed.nanny - WARNING - Worker process 14652 was killed by signal 15
distributed.nanny - WARNING - Worker process 28456 was killed by signal 15
distributed.nanny - WARNING - Worker process 11664 was killed by signal 15
distributed.nanny - WARNING - Worker process 15440 was killed by signal 15
distributed.nanny - WARNING - Worker process 35172 was killed by signal 15
distributed.nanny - WARNING - Worker process 26348 was killed by signal 15
distributed.nanny - WARNING - Worker process 26412 was killed by signal 15
distributed.nanny - WARNING - Worker process 21584 was killed by signal 15
distributed.nanny - WARNING - Worker process 23276 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 40936 was killed by signal 15
distributed.nanny - WARNING - Worker process 34716 was killed by signal 15
distributed.nanny - WARNING - Worker process 41560 was killed by signal 15
distributed.nanny - WARNING - Worker process 16060 was killed by signal 15
distributed.nanny - WARNING - Worker process 37980 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 2088 was killed by signal 15
distributed.nanny - WARNING - Worker process 36892 was killed by signal 15
distributed.nanny - WARNING - Worker process 1876 was killed by signal 15
distributed.nanny - WARNING - Worker process 32456 was killed by signal 15
distributed.nanny - WARNING - Worker process 34900 was killed by signal 15
distributed.nanny - WARNING - Worker process 34752 was killed by signal 15
distributed.nanny - WARNING - Worker process 29920 was killed by signal 15
distributed.nanny - WARNING - Worker process 27664 was killed by signal 15
distributed.nanny - WARNING - Worker process 26944 was killed by signal 15
distributed.nanny - WARNING - Worker process 26196 was killed by signal 15
distributed.nanny - WARNING - Worker process 30268 was killed by signal 15
distributed.nanny - WARNING - Worker process 9676 was killed by signal 15
distributed.nanny - WARNING - Worker process 40304 was killed by signal 15
distributed.nanny - WARNING - Worker process 9736 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 40980 was killed by signal 15
distributed.nanny - WARNING - Worker process 34476 was killed by signal 15
distributed.nanny - WARNING - Worker process 27248 was killed by signal 15
distributed.nanny - WARNING - Worker process 24340 was killed by signal 15
distributed.nanny - WARNING - Worker process 42980 was killed by signal 15
distributed.nanny - WARNING - Worker process 9196 was killed by signal 15
distributed.nanny - WARNING - Worker process 6112 was killed by signal 15
distributed.nanny - WARNING - Worker process 26368 was killed by signal 15
distributed.nanny - WARNING - Worker process 32336 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 39624 was killed by signal 15
distributed.nanny - WARNING - Worker process 27836 was killed by signal 15
distributed.nanny - WARNING - Worker process 5484 was killed by signal 15
distributed.nanny - WARNING - Worker process 40940 was killed by signal 15
distributed.nanny - WARNING - Worker process 40556 was killed by signal 15
distributed.nanny - WARNING - Worker process 32868 was killed by signal 15
distributed.nanny - WARNING - Worker process 33648 was killed by signal 15
distributed.nanny - WARNING - Worker process 38660 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 4580 was killed by signal 15
distributed.nanny - WARNING - Worker process 28204 was killed by signal 15
distributed.nanny - WARNING - Worker process 24136 was killed by signal 15
distributed.nanny - WARNING - Worker process 15160 was killed by signal 15
distributed.nanny - WARNING - Worker process 37820 was killed by signal 15
distributed.nanny - WARNING - Worker process 21088 was killed by signal 15
distributed.nanny - WARNING - Worker process 30456 was killed by signal 15
distributed.nanny - WARNING - Worker process 30700 was killed by signal 15
distributed.nanny - WARNING - Worker process 7300 was killed by signal 15
distributed.nanny - WARNING - Worker process 22952 was killed by signal 15
distributed.nanny - WARNING - Worker process 38028 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 38252 was killed by signal 15
distributed.nanny - WARNING - Worker process 9944 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 22664 was killed by signal 15
distributed.nanny - WARNING - Worker process 23876 was killed by signal 15
distributed.nanny - WARNING - Worker process 25360 was killed by signal 15
distributed.nanny - WARNING - Worker process 20348 was killed by signal 15
distributed.nanny - WARNING - Worker process 23224 was killed by signal 15
distributed.nanny - WARNING - Worker process 32532 was killed by signal 15
distributed.nanny - WARNING - Worker process 8152 was killed by signal 15
distributed.nanny - WARNING - Worker process 27656 was killed by signal 15
distributed.nanny - WARNING - Worker process 20556 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 36036 was killed by signal 15
distributed.nanny - WARNING - Worker process 5732 was killed by signal 15
distributed.nanny - WARNING - Worker process 30916 was killed by signal 15
distributed.nanny - WARNING - Worker process 30128 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 22452 was killed by signal 15
distributed.nanny - WARNING - Worker process 23496 was killed by signal 15
distributed.nanny - WARNING - Worker process 4196 was killed by signal 15
distributed.nanny - WARNING - Worker process 27580 was killed by signal 15
distributed.nanny - WARNING - Worker process 36320 was killed by signal 15
distributed.nanny - WARNING - Worker process 32432 was killed by signal 15
distributed.nanny - WARNING - Worker process 37808 was killed by signal 15
distributed.nanny - WARNING - Worker process 24056 was killed by signal 15
distributed.nanny - WARNING - Worker process 27808 was killed by signal 15
distributed.nanny - WARNING - Worker process 26400 was killed by signal 15
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Restarting worker
distributed.nanny - WARNING - Worker process 5696 was killed by signal 15
distributed.nanny - WARNING - Worker process 29328 was killed by signal 15
distributed.nanny - WARNING - Worker process 24788 was killed by signal 15
distributed.nanny - WARNING - Worker process 13124 was killed by signal 15
distributed.diskutils - WARNING - Found stale lock file and directory 'C:\\Users\\hoffmand\\dask-worker-space\\worker-fhuoxeh9', purging
distributed.diskutils - WARNING - Found stale lock file and directory 'C:\\Users\\hoffmand\\dask-worker-space\\worker-gybnbhub', purging
distributed.diskutils - WARNING - Found stale lock file and directory 'C:\\Users\\hoffmand\\dask-worker-space\\worker-i9c6pbx8', purging
distributed.diskutils - WARNING - Found stale lock file and directory 'C:\\Users\\hoffmand\\dask-worker-space\\worker-jdwmirzc', purging
distributed.diskutils - WARNING - Found stale lock file and directory 'C:\\Users\\hoffmand\\dask-worker-space\\worker-p10rx7az', purging
distributed.diskutils - WARNING - Found stale lock file and directory 'C:\\Users\\hoffmand\\dask-worker-space\\worker-taxo0tu3', purging
distributed.diskutils - WARNING - Found stale lock file and directory 'C:\\Users\\hoffmand\\dask-worker-space\\worker-_3fzltwp', purging
distributed.diskutils - ERROR - Failed to remove "[WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\hoffmand\\\\dask-worker-space\\\\worker-r6yj1ffx.dirlock'"
distributed.diskutils - ERROR - Failed to remove "[WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\hoffmand\\\\dask-worker-space\\\\worker-v7_efylx.dirlock'"
distributed.diskutils - ERROR - Failed to remove "[WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\hoffmand\\\\dask-worker-space\\\\worker-2qdow7ms.dirlock'"
distributed.diskutils - ERROR - Failed to remove "[WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\hoffmand\\\\dask-worker-space\\\\worker-77y9lvr9.dirlock'"
distributed.diskutils - ERROR - Failed to remove "[WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\hoffmand\\\\dask-worker-space\\\\worker-mjjsqsei.dirlock'"
distributed.diskutils - ERROR - Failed to remove "[WinError 5] Access is denied: 'C:\\\\Users\\\\hoffmand\\\\dask-worker-space\\\\worker-3cg91tc8.dirlock'"
distributed.diskutils - ERROR - Failed to remove "[WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\hoffmand\\\\dask-worker-space\\\\worker-eif4q87o.dirlock'"
distributed.diskutils - ERROR - Failed to remove "[WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\hoffmand\\\\dask-worker-space\\\\worker-eg647ilh.dirlock'"
distributed.diskutils - ERROR - Failed to remove "[WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\hoffmand\\\\dask-worker-space\\\\worker-9ncqzlo1.dirlock'"
distributed.diskutils - ERROR - Failed to remove "[WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\hoffmand\\\\dask-worker-space\\\\worker-ejhiutwq.dirlock'"
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
distributed.utils - ERROR - Worker failed to start
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\utils.py", line 238, in f
result[0] = yield make_coro()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1099, in run
value = future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1107, in run
yielded = self.gen.throw(*exc_info)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 163, in _start
yield [self._start_worker(**self.worker_kwargs) for i in range(n_workers)]
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1099, in run
value = future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 189, in _start_worker
raise gen.TimeoutError("Worker failed to start")
tornado.util.TimeoutError: Worker failed to start
tornado.application - ERROR - Exception in Future <Future finished exception=AssertionError({'address': 'tcp://127.0.0.1:62096', 'dir': 'C:\\Users\\hoffmand\\dask-worker-space\\worker-xift419g'},)> after timeout
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 936, in error_callback
future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1107, in run
yielded = self.gen.throw(*exc_info)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py", line 363, in start
self._wait_until_started())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1099, in run
value = future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py", line 471, in _wait_until_started
assert msg == 'started', msg
AssertionError: {'address': 'tcp://127.0.0.1:62096', 'dir': 'C:\\Users\\hoffmand\\dask-worker-space\\worker-xift419g'}
tornado.application - ERROR - Exception in Future <Future finished exception=AssertionError({'address': 'tcp://127.0.0.1:62104', 'dir': 'C:\\Users\\hoffmand\\dask-worker-space\\worker-0uy0anoj'},)> after timeout
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 936, in error_callback
future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1107, in run
yielded = self.gen.throw(*exc_info)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py", line 363, in start
self._wait_until_started())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1099, in run
value = future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py", line 471, in _wait_until_started
assert msg == 'started', msg
AssertionError: {'address': 'tcp://127.0.0.1:62104', 'dir': 'C:\\Users\\hoffmand\\dask-worker-space\\worker-0uy0anoj'}
---------------------------------------------------------------------------
TimeoutError Traceback (most recent call last)
<ipython-input-1-541dea1abdd2> in <module>()
1 from distributed import Client, LocalCluster
----> 2 cluster = LocalCluster()
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py in __init__(self, n_workers, threads_per_worker, processes, loop, start, ip, scheduler_port, silence_logs, diagnostics_port, services, worker_services, service_kwargs, asynchronous, **worker_kwargs)
125 self.worker_kwargs = worker_kwargs
126
--> 127 self.start(ip=ip, n_workers=n_workers)
128
129 clusters_to_close.add(self)
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py in start(self, **kwargs)
143 self._started = self._start(**kwargs)
144 else:
--> 145 sync(self.loop, self._start, **kwargs)
146
147 @gen.coroutine
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\utils.py in tornado.application - ERROR - Exception in Future <Future finished exception=AssertionError({'address': 'tcp://127.0.0.1:62112', 'dir': 'C:\\Users\\hoffmand\\dask-worker-space\\worker-mvlpzn11'},)> after timeout
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 936, in error_callback
future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1107, in run
yielded = self.gen.throw(*exc_info)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py", line 363, in start
self._wait_until_started())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1099, in run
value = future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py", line 471, in _wait_until_started
assert msg == 'started', msg
AssertionError: {'address': 'tcp://127.0.0.1:62112', 'dir': 'C:\\Users\\hoffmand\\dask-worker-space\\worker-mvlpzn11'}
sync(loop, func, *args, **kwargs)
252 e.wait(10)
253 if error[0]:
--> 254 six.reraise(*error[0])
255 else:
256 return result[0]
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\six.py in reraise(tp, value, tb)
691 if value.__traceback__ is not tb:
692 raise valuetornado.application - ERROR - Exception in Future <Future finished exception=AssertionError({'address': 'tcp://127.0.0.1:62124', 'dir': 'C:\\Users\\hoffmand\\dask-worker-space\\worker-145d7_hm'},)> after timeout
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 936, in error_callback
future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py", line 471, in _wait_until_started
assert msg == 'started', msg
AssertionError: {'address': 'tcp://127.0.0.1:62124', 'dir': 'C:\\Users\\hoffmand\\dask-worker-space\\worker-145d7_hm'}
.with_traceback(tb)
--> 693 raise value
694 finally:
695 value = None
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\utils.py in f()
236 yield gen.moment
237 thread_state.asynchronous = True
--> 238 result[0] = yield make_coro()
239 except Exception as exc:
240 logger.exception(exc)
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1097
1098 try:
-> 1099 value = future.result()
1100 except Exception:
1101 self.had_exception = True
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1105 if exc_info is not None:
1106 try:
-> 1107 yielded = self.gen.throw(*exc_info)
1108 finally:
1109 # Break up a reference to itself
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.pytornado.application - ERROR - Exception in Future <Future finished exception=AssertionError({'address': 'tcp://127.0.0.1:62128', 'dir': 'C:\\Users\\hoffmand\\dask-worker-space\\worker-f0xorv2r'},)> after timeout
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 936, in error_callback
future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py", line 471, in _wait_until_started
assert msg == 'started', msg
AssertionError: {'address': 'tcp://127.0.0.1:62128', 'dir': 'C:\\Users\\hoffmand\\dask-worker-space\\worker-f0xorv2r'}
in _start(self, ip, n_workers)
161 self.scheduler.start(scheduler_address)
162
--> 163 yield [self._start_worker(**self.worker_kwargs) for i in range(n_workers)]
164
165 self.status = 'running'
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1097
1098 try:
-> 1099 value = future.result()
1100 except Exception:
1101 self.had_exception = True
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in callback(f)
847 for f in children:
848 try:
--> 849 result_list.append(f.result())
850 except Exception as e:
851 if future.done():
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1111 exc_info = None
1112 else:
-> 1113 yielded = self.gen.send(value)
1114
1115 if stack_context._state.contexts is not orig_stack_contexts:
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py in _start_worker(self, death_timeout, **kwargs)
187 if w.status == 'closed':
188 self.workers.remove(w)
--> 189 raise gen.TimeoutError("Worker failed to start")
190
191 raise gen.Return(w)
TimeoutError: Worker failed to start
Second Error Message
distributed.utils - ERROR - {'address': 'tcp://127.0.0.1:62694', 'dir': 'C:\\Users\\hoffmand\\dask-worker-space\\worker-krpwx6ll'}
Traceback (most recent call last):
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\utils.py", line 238, in f
result[0] = yield make_coro()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1099, in run
value = future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1107, in run
yielded = self.gen.throw(*exc_info)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 163, in _start
yield [self._start_worker(**self.worker_kwargs) for i in range(n_workers)]
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1099, in run
value = future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 849, in callback
result_list.append(f.result())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1107, in run
yielded = self.gen.throw(*exc_info)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py", line 180, in _start_worker
yield w._start()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1099, in run
value = future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1107, in run
yielded = self.gen.throw(*exc_info)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py", line 155, in _start
response = yield self.instantiate()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1099, in run
value = future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1107, in run
yielded = self.gen.throw(*exc_info)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py", line 223, in instantiate
self.process.start()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1099, in run
value = future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1107, in run
yielded = self.gen.throw(*exc_info)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py", line 363, in start
self._wait_until_started())
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1099, in run
value = future.result()
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py", line 1113, in run
yielded = self.gen.send(value)
File "C:\Users\hoffmand\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py", line 471, in _wait_until_started
assert msg == 'started', msg
AssertionError: {'address': 'tcp://127.0.0.1:62694', 'dir': 'C:\\Users\\hoffmand\\dask-worker-space\\worker-krpwx6ll'}
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-1-65a3cf5a6c3a> in <module>()
1 from distributed import Client, LocalCluster
----> 2 cluster = LocalCluster(n_workers=24)
3 client = Client(cluster)
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py in __init__(self, n_workers, threads_per_worker, processes, loop, start, ip, scheduler_port, silence_logs, diagnostics_port, services, worker_services, service_kwargs, asynchronous, **worker_kwargs)
125 self.worker_kwargs = worker_kwargs
126
--> 127 self.start(ip=ip, n_workers=n_workers)
128
129 clusters_to_close.add(self)
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py in start(self, **kwargs)
143 self._started = self._start(**kwargs)
144 else:
--> 145 sync(self.loop, self._start, **kwargs)
146
147 @gen.coroutine
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\utils.py in sync(loop, func, *args, **kwargs)
252 e.wait(10)
253 if error[0]:
--> 254 six.reraise(*error[0])
255 else:
256 return result[0]
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\six.py in reraise(tp, value, tb)
691 if value.__traceback__ is not tb:
692 raise value.with_traceback(tb)
--> 693 raise value
694 finally:
695 value = None
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\utils.py in f()
236 yield gen.moment
237 thread_state.asynchronous = True
--> 238 result[0] = yield make_coro()
239 except Exception as exc:
240 logger.exception(exc)
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1097
1098 try:
-> 1099 value = future.result()
1100 except Exception:
1101 self.had_exception = True
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1105 if exc_info is not None:
1106 try:
-> 1107 yielded = self.gen.throw(*exc_info)
1108 finally:
1109 # Break up a reference to itself
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py in _start(self, ip, n_workers)
161 self.scheduler.start(scheduler_address)
162
--> 163 yield [self._start_worker(**self.worker_kwargs) for i in range(n_workers)]
164
165 self.status = 'running'
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1097
1098 try:
-> 1099 value = future.result()
1100 except Exception:
1101 self.had_exception = True
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in callback(f)
847 for f in children:
848 try:
--> 849 result_list.append(f.result())
850 except Exception as e:
851 if future.done():
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1105 if exc_info is not None:
1106 try:
-> 1107 yielded = self.gen.throw(*exc_info)
1108 finally:
1109 # Break up a reference to itself
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\deploy\local.py in _start_worker(self, death_timeout, **kwargs)
178 death_timeout=death_timeout,
179 silence_logs=self.silence_logs, **kwargs)
--> 180 yield w._start()
181
182 self.workers.append(w)
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1097
1098 try:
-> 1099 value = future.result()
1100 except Exception:
1101 self.had_exception = True
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1105 if exc_info is not None:
1106 try:
-> 1107 yielded = self.gen.throw(*exc_info)
1108 finally:
1109 # Break up a reference to itself
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py in _start(self, addr_or_port)
153
154 logger.info(' Start Nanny at: %r', self.address)
--> 155 response = yield self.instantiate()
156 if response == 'running':
157 assert self.worker_address
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1097
1098 try:
-> 1099 value = future.result()
1100 except Exception:
1101 self.had_exception = True
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1105 if exc_info is not None:
1106 try:
-> 1107 yielded = self.gen.throw(*exc_info)
1108 finally:
1109 # Break up a reference to itself
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py in instantiate(self, comm)
221 result = yield gen.with_timeout(
222 timedelta(seconds=self.death_timeout),
--> 223 self.process.start()
224 )
225 except gen.TimeoutError:
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1097
1098 try:
-> 1099 value = future.result()
1100 except Exception:
1101 self.had_exception = True
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1105 if exc_info is not None:
1106 try:
-> 1107 yielded = self.gen.throw(*exc_info)
1108 finally:
1109 # Break up a reference to itself
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py in start(self)
361 if self.status == 'starting':
362 yield gen.with_timeout(timedelta(seconds=5),
--> 363 self._wait_until_started())
364 except gen.TimeoutError:
365 logger.info("Failed to start worker process. Restarting")
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1097
1098 try:
-> 1099 value = future.result()
1100 except Exception:
1101 self.had_exception = True
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\tornado\gen.py in run(self)
1111 exc_info = None
1112 else:
-> 1113 yielded = self.gen.send(value)
1114
1115 if stack_context._state.contexts is not orig_stack_contexts:
~\AppData\Local\Continuum\Anaconda3\lib\site-packages\distributed\nanny.py in _wait_until_started(self)
469 try:
470 msg = self.init_result_q.get_nowait()
--> 471 assert msg == 'started', msg
472 return
473 except Empty:
AssertionError: {'address': 'tcp://127.0.0.1:62694', 'dir': 'C:\\Users\\hoffmand\\dask-worker-space\\worker-krpwx6ll'}
Issue Analytics
- State:
- Created 5 years ago
- Comments:13 (8 by maintainers)
Top Results From Across the Web
python - Multiprocessing: use only the physical cores?
Suppose I have a CPU with 4 physical cores, each with two logical cores. My system has enough memory to accommodate 4 instances...
Read more >432 startup failure post-mortems
It's hard to say goodbye. Find out how 432 startups have failed and why in the words of their investors and founders.
Read more >What Is CPU? Cores, Multithreading, & vCPU
In this analysis, we'll take a look at the functions of a CPU, discuss the roles and differences between physical cores and logical...
Read more >Virtual CPU Limitations
Be aware of the following limitations: A virtual machine cannot have more virtual CPUs than the number of logical cores on the host....
Read more >How can our company determine its core competencies?
Typically, a core competency refers to a company's set of skills or experience in some activity, rather than physical or financial assets.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
OK, that is very helpful. I appreciate your engagement here. Hopefully by working from one of those branches you’ll be find short term. I’ll merge one of those two for the next release, probably within a week or two.
Short term I’ve gone with the other approach. There is a somewhat-rare failure on UNIX-based systems otherwise. Hopefully we’ll be able to track it down soon.
Thank you again for your collaboration here. Closing.