BUG fresh install on OSX conda env with pip gives a segfault

We are using scikit-learn our benchopt package and we came across a weird behavior when installing the 1.0 release on OSX with pip in a conda env in the CI, where the import of scikit-learn causes a segfault.

After a bit of debugging, we were able to pinpoint that it comes from an interation with installing numba in the conda env beforehand. Here are the PR where we investigated this https://github.com/benchopt/benchOpt/pull/211 and the error log with a minimal reproduction: https://github.com/benchopt/benchOpt/runs/3733088581?check_suite_focus=true

Step to reproduce:

conda create -n test_env -c conda-forge python=3.8 numpy scipy numba
conda activate test_env
pip install scikit-learn
python -c 'from sklearn.linear_model import Lasso'

This results in:

line 4: 2841 Segmentation fault: 11  python -c 'from sklearn.linear_model import Lasso'

We fixed our issue by switching to conda install which is probably safer.

[thomasjpfan]

I can reproduce on OSX intel, using this test file:

from sklearn.linear_model import Lasso

with lldb python test.py and running

Process 6123 stopped
* thread #2, stop reason = EXC_BAD_ACCESS (code=1, address=0x8)
    frame #0: 0x000000010179b189 libomp.dylib`void __kmp_suspend_64<false, true>(int, kmp_flag_64<false, true>*) + 36
libomp.dylib`__kmp_suspend_64<false, true>:
->  0x10179b189 <+36>: movq   (%rax,%rcx,8), %r13
    0x10179b18d <+40>: movq   %r13, %rdi
    0x10179b190 <+43>: callq  0x10179a9da               ; __kmp_suspend_initialize_thread
    0x10179b195 <+48>: movq   %r13, %rdi

Looking into a fix.

[thomasjpfan]

Since the nightly wheels are built openmp 11 on osx now, one can check to see if the import works with the nightly build:

pip install -i https://pypi.anaconda.org/scipy-wheels-nightly/simple scikit-learn

The main branch and 1.0.X has not diverged too much given 1.0 was release not too long ago.