BUG: Container build started failing since scipy 1.7.2
See original GitHub issueDescribe your issue.
For one of my services running in K8s I have my base image with Debian GNU/Linux 10 (buster)
and Python 3.7.5
installed into this image. On top of that, at every push to repo, my CI/CD pipeline builds another image with different Python packages including scipy
. Everything had worked fine until I pushed new code today which apparently triggered installing scipy==1.7.2
into my Docker container for the first time. It used to be 1.7.1
before that. The version of scipy
used to get updated every time because it had not been put explicitly into requirements.txt
before. It had only been installed as a dependency of another Python package.
After seeing my CI/CD pipeline failing I fixed scipy
to 1.7.1
in requirements.txt
file, which is run by my Dockerfile
. And everything started working for me again.
So the problem is kinda fixed for me now, but still the 1.7.2
release page here does not inform of this kind of possible problems after version update. And all this is quite a bit more problematic than I could expect from a minor bug fix release like this one.
Reproducing Code Example
This code broke:
`RUN pip3 install -r requirements.txt`
Error message
Here's my GitLab runner's logs:
wheels for collected packages: numpy, scipy
Running setup.py bdist_wheel for numpy: started
Running setup.py bdist_wheel for numpy: still running...
Running setup.py bdist_wheel for numpy: still running...
Running setup.py bdist_wheel for numpy: still running...
Running setup.py bdist_wheel for numpy: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/32/74/29/a670d29ea068e3ff91889eed2a6b590d0b39af39ec77d49607
Running setup.py bdist_wheel for scipy: started
Running setup.py bdist_wheel for scipy: still running...
Running setup.py bdist_wheel for scipy: still running...
Running setup.py bdist_wheel for scipy: finished with status 'error'
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-1s08xzuz/scipy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-mr6ylhmu --python-tag cp37:
Running from SciPy source directory.
Cythonizing sources
Running scipy/linalg/_generate_pyx.py
Running scipy/stats/_generate_pyx.py
Running scipy/special/_generate_pyx.py
Processing scipy/spatial/_voronoi.pyx
Processing scipy/spatial/ckdtree.pyx
Processing scipy/spatial/qhull.pyx
Processing scipy/spatial/_hausdorff.pyx
Processing scipy/spatial/transform/rotation.pyx
Processing scipy/linalg/cython_blas.pyx
Processing scipy/linalg/_decomp_update.pyx.in
Processing scipy/linalg/_matfuncs_sqrtm_triu.pyx
Processing scipy/linalg/cython_lapack.pyx
Processing scipy/linalg/_solve_toeplitz.pyx
Processing scipy/fftpack/convolve.pyx
Processing scipy/signal/_sosfilt.pyx
Processing scipy/signal/_peak_finding_utils.pyx
Processing scipy/signal/_upfirdn_apply.pyx
Processing scipy/signal/_max_len_seq_inner.pyx
Processing scipy/signal/_spectral.pyx
Processing scipy/cluster/_optimal_leaf_ordering.pyx
Processing scipy/cluster/_hierarchy.pyx
Processing scipy/cluster/_vq.pyx
Processing scipy/optimize/_group_columns.pyx
Processing scipy/optimize/_bglu_dense.pyx
Processing scipy/optimize/_trlib/_trlib.pyx
Processing scipy/optimize/cython_optimize/_zeros.pyx.in
Processing scipy/optimize/_highs/cython/src/_highs_constants.pyx
Processing scipy/optimize/_highs/cython/src/_highs_wrapper.pyx
Processing scipy/optimize/_lsq/givens_elimination.pyx
Processing scipy/sparse/_csparsetools.pyx.in
Processing scipy/sparse/csgraph/_matching.pyx
Processing scipy/sparse/csgraph/_min_spanning_tree.pyx
Processing scipy/sparse/csgraph/_flow.pyx
Processing scipy/sparse/csgraph/_shortest_path.pyx
Processing scipy/sparse/csgraph/_traversal.pyx
Processing scipy/sparse/csgraph/_reordering.pyx
Processing scipy/sparse/csgraph/_tools.pyx
Processing scipy/ndimage/src/_ni_label.pyx
Processing scipy/ndimage/src/_cytest.pyx
Processing scipy/stats/_sobol.pyx
Processing scipy/stats/_stats.pyx
Processing scipy/stats/_qmc_cy.pyx
Processing scipy/stats/biasedurn.pyx
Processing scipy/stats/_boost/src/binom_ufunc.pyx
Processing scipy/stats/_boost/src/nbinom_ufunc.pyx
Processing scipy/stats/_boost/src/beta_ufunc.pyx
Processing scipy/special/_ellip_harm_2.pyx
Processing scipy/special/_test_round.pyx
Processing scipy/special/_comb.pyx
Processing scipy/special/cython_special.pyx
Processing scipy/special/_ufuncs_cxx.pyx
Processing scipy/special/_ufuncs.pyx
Processing scipy/interpolate/interpnd.pyx
Processing scipy/interpolate/_ppoly.pyx
Processing scipy/interpolate/_bspl.pyx
Processing scipy/_lib/_ccallback_c.pyx
Processing scipy/_lib/messagestream.pyx
Processing scipy/_lib/_test_deprecation_call.pyx
Processing scipy/_lib/_test_deprecation_def.pyx
Processing scipy/io/matlab/mio_utils.pyx
Processing scipy/io/matlab/mio5_utils.pyx
Processing scipy/io/matlab/streams.pyx
warning: _cython_special_custom.pxi:9:8: Unreachable code
warning: _cython_special_custom.pxi:13:4: Unreachable code
warning: _cython_special_custom.pxi:21:8: Unreachable code
warning: _cython_special_custom.pxi:25:4: Unreachable code
warning: _cython_special_custom.pxi:33:8: Unreachable code
warning: _cython_special_custom.pxi:37:4: Unreachable code
warning: _cython_special_custom.pxi:45:8: Unreachable code
warning: _cython_special_custom.pxi:49:4: Unreachable code
lapack_opt_info:
lapack_mkl_info:
customize UnixCCompiler
libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
openblas_lapack_info:
customize UnixCCompiler
customize UnixCCompiler
libraries openblas not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
openblas_clapack_info:
customize UnixCCompiler
customize UnixCCompiler
libraries openblas,lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
atlas_3_10_threads_info:
Setting PTATLAS=ATLAS
customize UnixCCompiler
libraries lapack_atlas not found in /usr/local/lib
customize UnixCCompiler
libraries tatlas,tatlas not found in /usr/local/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib
customize UnixCCompiler
libraries tatlas,tatlas not found in /usr/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
customize UnixCCompiler
libraries tatlas,tatlas not found in /usr/lib/x86_64-linux-gnu
<class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
NOT AVAILABLE
atlas_3_10_info:
customize UnixCCompiler
libraries lapack_atlas not found in /usr/local/lib
customize UnixCCompiler
libraries satlas,satlas not found in /usr/local/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib
customize UnixCCompiler
libraries satlas,satlas not found in /usr/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
customize UnixCCompiler
libraries satlas,satlas not found in /usr/lib/x86_64-linux-gnu
<class 'numpy.distutils.system_info.atlas_3_10_info'>
NOT AVAILABLE
atlas_threads_info:
Setting PTATLAS=ATLAS
customize UnixCCompiler
libraries lapack_atlas not found in /usr/local/lib
customize UnixCCompiler
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib
customize UnixCCompiler
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
customize UnixCCompiler
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/x86_64-linux-gnu
<class 'numpy.distutils.system_info.atlas_threads_info'>
NOT AVAILABLE
atlas_info:
customize UnixCCompiler
libraries lapack_atlas not found in /usr/local/lib
customize UnixCCompiler
libraries f77blas,cblas,atlas not found in /usr/local/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib
customize UnixCCompiler
libraries f77blas,cblas,atlas not found in /usr/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
customize UnixCCompiler
libraries f77blas,cblas,atlas not found in /usr/lib/x86_64-linux-gnu
<class 'numpy.distutils.system_info.atlas_info'>
NOT AVAILABLE
accelerate_info:
NOT AVAILABLE
/tmp/pip-build-env-fcsprb6_/lib/python3.7/site-packages/numpy/distutils/system_info.py:639: UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
self.calc_info()
lapack_info:
customize UnixCCompiler
libraries lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
/tmp/pip-build-env-fcsprb6_/lib/python3.7/site-packages/numpy/distutils/system_info.py:639: UserWarning:
Lapack (http://www.netlib.org/lapack/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [lapack]) or by setting
the LAPACK environment variable.
self.calc_info()
lapack_src_info:
NOT AVAILABLE
/tmp/pip-build-env-fcsprb6_/lib/python3.7/site-packages/numpy/distutils/system_info.py:639: UserWarning:
Lapack (http://www.netlib.org/lapack/) sources not found.
Directories to search for the sources can be specified in the
numpy/distutils/site.cfg file (section [lapack_src]) or by setting
the LAPACK_SRC environment variable.
self.calc_info()
NOT AVAILABLE
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-1s08xzuz/scipy/setup.py", line 631, in <module>
setup_package()
File "/tmp/pip-install-1s08xzuz/scipy/setup.py", line 627, in setup_package
setup(**metadata)
File "/tmp/pip-build-env-fcsprb6_/lib/python3.7/site-packages/numpy/distutils/core.py", line 137, in setup
config = configuration()
File "/tmp/pip-install-1s08xzuz/scipy/setup.py", line 529, in configuration
raise NotFoundError(msg)
numpy.distutils.system_info.NotFoundError: No BLAS/LAPACK libraries found.
To build Scipy from sources, BLAS & LAPACK libraries need to be installed.
See site.cfg.example in the Scipy source directory and
https://docs.scipy.org/doc/scipy/reference/building/index.html for details.
----------------------------------------
Failed building wheel for scipy
Running setup.py clean for scipy
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-1s08xzuz/scipy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" clean --all:
`setup.py clean` is not supported, use one of the following instead:
- `git clean -xdf` (cleans all files)
- `git clean -Xdf` (cleans all versioned files, doesn't touch
files that aren't checked into the git repo)
Add `--force` to your command to use it anyway if you must (unsupported).
----------------------------------------
Failed cleaning build dir for scipy
Successfully built numpy
Failed to build scipy
Installing collected packages: Werkzeug, MarkupSafe, Jinja2, zipp, typing-extensions, importlib-metadata, click, itsdangerous, Flask, gunicorn, psycopg2-binary, aniso8601, pytz, attrs, pyrsistent, importlib-resources, jsonschema, flask-restplus, numpy, pandas, chardet, urllib3, idna, certifi, requests, arrow, pyckson, py-zabbix, tzlocal, APScheduler, croniter, greenlet, sqlalchemy, joblib, scipy, threadpoolctl, scikit-learn
Running setup.py install for scipy: started
Running setup.py install for scipy: finished with status 'error'
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-1s08xzuz/scipy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-ydpeax0t/install-record.txt --single-version-externally-managed --compile:
Note: for reliable uninstall behaviour and dependency installation
and uninstallation, please use pip instead of using
`setup.py install`:
- `pip install .` (from a git repo or downloaded source
release)
- `pip install scipy` (last SciPy release on PyPI)
Running from SciPy source directory.
Cythonizing sources
scipy/linalg/_generate_pyx.py: all files up-to-date
scipy/special/_generate_pyx.py: all files up-to-date
Running scipy/linalg/_generate_pyx.py
Running scipy/stats/_generate_pyx.py
Running scipy/special/_generate_pyx.py
scipy/spatial/_voronoi.pyx has not changed
scipy/stats/_boost/src/binom_ufunc.pyx has not changed
scipy/stats/_boost/src/nbinom_ufunc.pyx has not changed
scipy/stats/_boost/src/beta_ufunc.pyx has not changed
scipy/special/_ellip_harm_2.pyx has not changed
scipy/special/_test_round.pyx has not changed
scipy/special/_comb.pyx has not changed
scipy/special/cython_special.pyx has not changed
scipy/special/_ufuncs_cxx.pyx has not changed
scipy/special/_ufuncs.pyx has not changed
scipy/interpolate/interpnd.pyx has not changed
scipy/interpolate/_ppoly.pyx has not changed
scipy/interpolate/_bspl.pyx has not changed
scipy/_lib/_ccallback_c.pyx has not changed
scipy/_lib/messagestream.pyx has not changed
scipy/_lib/_test_deprecation_call.pyx has not changed
scipy/_lib/_test_deprecation_def.pyx has not changed
scipy/io/matlab/mio_utils.pyx has not changed
scipy/cluster/_vq.pyx has not changed
scipy/optimize/_group_columns.pyx has not changed
scipy/optimize/_bglu_dense.pyx has not changed
scipy/optimize/_trlib/_trlib.pyx has not changed
scipy/optimize/cython_optimize/_zeros.pyx.in has not changed
scipy/optimize/_highs/cython/src/_highs_constants.pyx has not changed
scipy/optimize/_highs/cython/src/_highs_wrapper.pyx has not changed
scipy/optimize/_lsq/givens_elimination.pyx has not changed
scipy/sparse/_csparsetools.pyx.in has not changed
scipy/sparse/csgraph/_matching.pyx has not changed
scipy/sparse/csgraph/_min_spanning_tree.pyx has not changed
scipy/sparse/csgraph/_flow.pyx has not changed
scipy/sparse/csgraph/_shortest_path.pyx has not changed
scipy/sparse/csgraph/_traversal.pyx has not changed
scipy/sparse/csgraph/_reordering.pyx has not changed
scipy/sparse/csgraph/_tools.pyx has not changed
scipy/ndimage/src/_ni_label.pyx has not changed
scipy/ndimage/src/_cytest.pyx has not changed
scipy/stats/_sobol.pyx has not changed
scipy/stats/_stats.pyx has not changed
scipy/stats/_qmc_cy.pyx has not changed
scipy/stats/biasedurn.pyx has not changed
scipy/spatial/ckdtree.pyx has not changed
scipy/spatial/qhull.pyx has not changed
scipy/spatial/_hausdorff.pyx has not changed
scipy/spatial/transform/rotation.pyx has not changed
scipy/linalg/cython_blas.pyx has not changed
scipy/linalg/_decomp_update.pyx.in has not changed
scipy/linalg/_matfuncs_sqrtm_triu.pyx has not changed
scipy/linalg/cython_lapack.pyx has not changed
scipy/linalg/_solve_toeplitz.pyx has not changed
scipy/fftpack/convolve.pyx has not changed
scipy/signal/_sosfilt.pyx has not changed
scipy/signal/_peak_finding_utils.pyx has not changed
scipy/signal/_upfirdn_apply.pyx has not changed
scipy/signal/_max_len_seq_inner.pyx has not changed
scipy/signal/_spectral.pyx has not changed
scipy/cluster/_optimal_leaf_ordering.pyx has not changed
scipy/cluster/_hierarchy.pyx has not changed
scipy/io/matlab/mio5_utils.pyx has not changed
scipy/io/matlab/streams.pyx has not changed
lapack_opt_info:
lapack_mkl_info:
customize UnixCCompiler
libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
openblas_lapack_info:
customize UnixCCompiler
customize UnixCCompiler
libraries openblas not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
openblas_clapack_info:
customize UnixCCompiler
customize UnixCCompiler
libraries openblas,lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
atlas_3_10_threads_info:
Setting PTATLAS=ATLAS
customize UnixCCompiler
libraries lapack_atlas not found in /usr/local/lib
customize UnixCCompiler
libraries tatlas,tatlas not found in /usr/local/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib
customize UnixCCompiler
libraries tatlas,tatlas not found in /usr/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
customize UnixCCompiler
libraries tatlas,tatlas not found in /usr/lib/x86_64-linux-gnu
<class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
NOT AVAILABLE
atlas_3_10_info:
customize UnixCCompiler
libraries lapack_atlas not found in /usr/local/lib
customize UnixCCompiler
libraries satlas,satlas not found in /usr/local/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib
customize UnixCCompiler
libraries satlas,satlas not found in /usr/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
customize UnixCCompiler
libraries satlas,satlas not found in /usr/lib/x86_64-linux-gnu
<class 'numpy.distutils.system_info.atlas_3_10_info'>
NOT AVAILABLE
atlas_threads_info:
Setting PTATLAS=ATLAS
customize UnixCCompiler
libraries lapack_atlas not found in /usr/local/lib
customize UnixCCompiler
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib
customize UnixCCompiler
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
customize UnixCCompiler
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/x86_64-linux-gnu
<class 'numpy.distutils.system_info.atlas_threads_info'>
NOT AVAILABLE
atlas_info:
customize UnixCCompiler
libraries lapack_atlas not found in /usr/local/lib
customize UnixCCompiler
libraries f77blas,cblas,atlas not found in /usr/local/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib
customize UnixCCompiler
libraries f77blas,cblas,atlas not found in /usr/lib
customize UnixCCompiler
libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
customize UnixCCompiler
libraries f77blas,cblas,atlas not found in /usr/lib/x86_64-linux-gnu
<class 'numpy.distutils.system_info.atlas_info'>
NOT AVAILABLE
accelerate_info:
NOT AVAILABLE
/tmp/pip-build-env-fcsprb6_/lib/python3.7/site-packages/numpy/distutils/system_info.py:639: UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
self.calc_info()
lapack_info:
customize UnixCCompiler
libraries lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
/tmp/pip-build-env-fcsprb6_/lib/python3.7/site-packages/numpy/distutils/system_info.py:639: UserWarning:
Lapack (http://www.netlib.org/lapack/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [lapack]) or by setting
the LAPACK environment variable.
self.calc_info()
lapack_src_info:
NOT AVAILABLE
/tmp/pip-build-env-fcsprb6_/lib/python3.7/site-packages/numpy/distutils/system_info.py:639: UserWarning:
Lapack (http://www.netlib.org/lapack/) sources not found.
Directories to search for the sources can be specified in the
numpy/distutils/site.cfg file (section [lapack_src]) or by setting
the LAPACK_SRC environment variable.
self.calc_info()
NOT AVAILABLE
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-1s08xzuz/scipy/setup.py", line 631, in <module>
setup_package()
File "/tmp/pip-install-1s08xzuz/scipy/setup.py", line 627, in setup_package
setup(**metadata)
File "/tmp/pip-build-env-fcsprb6_/lib/python3.7/site-packages/numpy/distutils/core.py", line 137, in setup
config = configuration()
File "/tmp/pip-install-1s08xzuz/scipy/setup.py", line 529, in configuration
raise NotFoundError(msg)
numpy.distutils.system_info.NotFoundError: No BLAS/LAPACK libraries found.
To build Scipy from sources, BLAS & LAPACK libraries need to be installed.
See site.cfg.example in the Scipy source directory and
https://docs.scipy.org/doc/scipy/reference/building/index.html for details.
----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-1s08xzuz/scipy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-ydpeax0t/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-1s08xzuz/scipy/
The command '/bin/sh -c pip3 install -r requirements.txt' returned a non-zero code: 1
Cleaning up file based variables 00:00
ERROR: Job failed: command terminated with exit code 1
SciPy/NumPy/Python version information
1.7.2
Issue Analytics
- State:
- Created 2 years ago
- Comments:9 (7 by maintainers)
Hi guys, Sorry I’m still not answering your initial question, @tupui . I don’t have immediate access to the base image’s repo where all OS and Python stuff get installed. I requested access to it. Should get it today or tomorrow probably. Yeah, it’s weird that this BLAS problem is appearing with the introduction of
1.7.2
. Will try upgrading pip though. Thanks for the advice, @andyfaff .So I got access to the base image’s repo, and it’s actually a rather evolved chain of images. I don’t think you want to bother yourself with looking into all those Dockerfiles. I tried upgrading pip in my project’s Dockerfile and that did really work. I don’t have to pin
scipy
to1.7.1
anymore. So this may remain closed. Thanks a lot for your support, guys!