Tests test_theil_sen_parallel and test_multi_output_classification_partial_fit_parallelism hang on Windows

Description

Install scikit-learn 0.20.0 on Windows as follows:

conda create -n hang python=3.6.5
pip install scikit-learn==0.20.0
pip install pytest

Steps/Code to Reproduce

The following two individual test runs hang (never finish, and remain uninterruptable):

pytest -v --pyarg sklearn.tests.test_multioutput::test_multi_output_classification_partial_fit_parallelism

pytest -v --pyarg sklearn.linear_model.tests.test_theil_sen::test_theil_sen_parallel

Expected Results

Expecting them to pass as they do on Linux, or skipped in the distribution

Actual Results

Tests hang in both pip installed scikit-learn, as well in scikit-learn installed via conda itself.

Reproduced on “Windows Server 2012 R2 Standard”.

Versions

>pip list
Package        Version
-------------- ---------
atomicwrites   1.2.1
attrs          18.2.0
certifi        2018.8.24
colorama       0.3.9
more-itertools 4.3.0
numpy          1.15.2
pip            10.0.1
pluggy         0.7.1
py             1.6.0
pytest         3.8.2
scikit-learn   0.20.0
scipy          1.1.0
setuptools     40.2.0
six            1.11.0
wheel          0.31.1
wincertstore   0.2

@ogrisel

[oleksandr-pavlyk]

I volunteer to test fixes for free 😃

[tomMoral]

but more cores would still trigger the bug, even with python fixed, right? EC2 has c5.18xlarge with 72 cores for $0.70/h.

Note that this problem only exists on windows, so big instances are more around 5/10$ per hour.

But this should still be fixed in loky.