Unstable LogisticRegression with saga solver l1 penalty under macOS

======================================================================
FAIL: sklearn.linear_model.tests.test_logistic.test_logreg_l1_sparse_data
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/travis/build/MacPython/scikit-learn-wheels/venv/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/travis/build/MacPython/scikit-learn-wheels/venv/lib/python3.6/site-packages/sklearn/linear_model/tests/test_logistic.py", line 961, in test_logreg_l1_sparse_data
    assert_array_almost_equal(lr_saga.coef_, lr_liblinear.coef_)
  File "/Users/travis/build/MacPython/scikit-learn-wheels/venv/lib/python3.6/site-packages/numpy/testing/utils.py", line 962, in assert_array_almost_equal
    precision=decimal)
  File "/Users/travis/build/MacPython/scikit-learn-wheels/venv/lib/python3.6/site-packages/numpy/testing/utils.py", line 778, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals
(mismatch 64.0%)
 x: array([[ -128.209824,  -990.760917,     0.      ,     0.      ,
            0.      ,   783.90525 ,  1345.642841,     0.      ,
           34.880916,  -287.813868,  -364.684306,  -668.47532 ,...
 y: array([[-0.764085, -0.29863 ,  0.      ,  0.269297,  0.      ,  0.      ,
         2.629204,  0.      ,  0.      ,  0.      , -0.450364, -0.61554 ,
         0.      , -0.262512,  0.161514,  0.      ,  0.314769,  0.      ,...
======================================================================
FAIL: sklearn.linear_model.tests.test_logistic.test_saga_vs_liblinear
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/travis/build/MacPython/scikit-learn-wheels/venv/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/travis/build/MacPython/scikit-learn-wheels/venv/lib/python3.6/site-packages/sklearn/linear_model/tests/test_logistic.py", line 1144, in test_saga_vs_liblinear
    assert_array_almost_equal(saga.coef_, liblinear.coef_, 3)
  File "/Users/travis/build/MacPython/scikit-learn-wheels/venv/lib/python3.6/site-packages/numpy/testing/utils.py", line 962, in assert_array_almost_equal
    precision=decimal)
  File "/Users/travis/build/MacPython/scikit-learn-wheels/venv/lib/python3.6/site-packages/numpy/testing/utils.py", line 778, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 3 decimals
(mismatch 100.0%)
 x: array([[ -105.601, -1199.326,  2377.574,   479.408]])
 y: array([[ 0.   , -1.06 ,  1.222,  0.   ]])

numpy and scipy are imported from the following wheels from PyPI:

• numpy-1.11.3-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (at buil time)

• numpy-1.13.1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (at test time)

• scipy-0.19.1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl

• Cython-0.25.2-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl

The full log of the build is here:

https://s3.amazonaws.com/archive.travis-ci.org/jobs/253231280/log.txt?X-Amz-Expires=30&X-Amz-Date=20170713T160513Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJRYRXRSVGNKPKO5A/20170713/us-east-1/s3/aws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=06e6866cac7abfc10cc67aa061a0e45b8db234c71a5997c67d8cc63828838df2

The configuration scripts of this build and test run can be found here:

https://github.com/MacPython/scikit-learn-wheels

A similar problem happened with Python 2.7 on macOS:

https://travis-ci.org/MacPython/scikit-learn-wheels/jobs/253231277

but not with python 3.4 for some reason:

https://travis-ci.org/MacPython/scikit-learn-wheels/builds/253231260

The same tests pass under Linux and Windows both 64 and 32 bit for all supported versions of Python.

Maybe this is a problem caused by the Apple Accelerate implementation of BLAS.

[jnothman]

I woke up this morning realising why boundscheck wouldn’t work. It’s a pointer, not an array.

[jnothman]

Should Cython have found this for me if I set boundscheck=True? It didn’t, but maybe that’s just because it was failing to raise the exception with the gil off. Should we be regularly running all our Cython tests with boundscheck enabled and gil fixed to on?