scipy links to gfortran libraries with intel compilers.

Scipy built with icc/ifort links to gfortran libraries instead of intel fortran libraries. (intel compilers are used as per [1].) First reported here.

d_int_val comes from the libifcore library and should be linked. Instead gfortran libraries get linked and this results in missing symbols. Adding fcflags: -lifcore -fPIC doesn’t help either.

This happens because scipy thinks it’s working with G95 compiler and gives -ffixed-form flag which ifort does not understand. Doesn’t setting -fcompiler=intelem force the use of intel compiler ?

Though I’m not sure, I think scipy understands that it’s using icc because some intel libraries are linked as shown by ldd.

Error message:

[sajid@xrmlite ~]$ python
Python 3.6.6 (default, Jul 19 2018, 15:02:18)
[GCC Intel(R) C++ gcc 7.3 mode] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> import scipy
>>> import scipy.optimize
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/intel-18.0.3/py-scipy-1.1.0-s5rupidaz3anzojgs6t6mzqbbbonfsg2/lib/python3.6/site-packages/scipy/optimize/__init__.py", line 270, in <module>
    from .optimize import *
  File "/home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/intel-18.0.3/py-scipy-1.1.0-s5rupidaz3anzojgs6t6mzqbbbonfsg2/lib/python3.6/site-packages/scipy/optimize/optimize.py", line 37, in <module>
    from .linesearch import (line_search_wolfe1, line_search_wolfe2,
  File "/home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/intel-18.0.3/py-scipy-1.1.0-s5rupidaz3anzojgs6t6mzqbbbonfsg2/lib/python3.6/site-packages/scipy/optimize/linesearch.py", line 18, in <module>
    from scipy.optimize import minpack2
ImportError: /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/intel-18.0.3/py-scipy-1.1.0-s5rupidaz3anzojgs6t6mzqbbbonfsg2/lib/python3.6/site-packages/scipy/optimize/minpack2.cpython-36m-x86_64-linux-gnu.so: undefined symbol: for_cpystr
>>> import scipy.integrate
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/intel-18.0.3/py-scipy-1.1.0-s5rupidaz3anzojgs6t6mzqbbbonfsg2/lib/python3.6/site-packages/scipy/integrate/__init__.py", line 89, in <module>
    from .quadrature import *
  File "/home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/intel-18.0.3/py-scipy-1.1.0-s5rupidaz3anzojgs6t6mzqbbbonfsg2/lib/python3.6/site-packages/scipy/integrate/quadrature.py", line 10, in <module>
    from scipy.special import roots_legendre
  File "/home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/intel-18.0.3/py-scipy-1.1.0-s5rupidaz3anzojgs6t6mzqbbbonfsg2/lib/python3.6/site-packages/scipy/special/__init__.py", line 640, in <module>
    from ._ufuncs import *
ImportError: /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/intel-18.0.3/py-scipy-1.1.0-s5rupidaz3anzojgs6t6mzqbbbonfsg2/lib/python3.6/site-packages/scipy/special/_ufuncs.cpython-36m-x86_64-linux-gnu.so: undefined symbol: d_int_val
>>>

Link info for scipy :

[sajid@xrmlite special]$ ldd _ufuncs.cpython-36m-x86_64-linux-gnu.so
        linux-vdso.so.1 =>  (0x00007ffc81bcd000)
        libmkl_rt.so => /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/intel-18.0.3/intel-mkl-2018.2.199-y427ptu5mgmk34rjc4rsfhapxyf4j2fb/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_rt.so (0x00002b3909b48000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b390a1e2000)
        libpython3.6m.so.1.0 => /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/intel-18.0.3/python-3.6.6-iick7mjya7ytfu3bbytes7w7sr6wag7d/lib/libpython3.6m.so.1.0 (0x00002b390a3fe000)
        libgfortran.so.4 => /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/gcc-7.3.0-xyzezhjmbiebkjfoakso464rhfshlkyq/lib64/libgfortran.so.4 (0x00002b390aa06000)
        libm.so.6 => /lib64/libm.so.6 (0x00002b390add9000)
        libgcc_s.so.1 => /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/gcc-7.3.0-xyzezhjmbiebkjfoakso464rhfshlkyq/lib64/libgcc_s.so.1 (0x00002b390b0db000)
        libquadmath.so.0 => /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/gcc-7.3.0-xyzezhjmbiebkjfoakso464rhfshlkyq/lib64/libquadmath.so.0 (0x00002b390b2f3000)
        libc.so.6 => /lib64/libc.so.6 (0x00002b390b532000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00002b390b8f5000)
        /lib64/ld-linux-x86-64.so.2 (0x000055a4d43c2000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00002b390bafa000)
        libimf.so => /opt/intel/compilers_and_libraries_2018.3.222/linux/compiler/lib/intel64_lin/libimf.so (0x00002b390bcfe000)
        libsvml.so => /opt/intel/compilers_and_libraries_2018.3.222/linux/compiler/lib/intel64_lin/libsvml.so (0x00002b390c291000)
        libirng.so => /opt/intel/compilers_and_libraries_2018.3.222/linux/compiler/lib/intel64_lin/libirng.so (0x00002b390db9f000)
        libintlc.so.5 => /opt/intel/compilers_and_libraries_2018.3.222/linux/compiler/lib/intel64_lin/libintlc.so.5 (0x00002b390df14000)
~

Link info for intel-python-scipy :

[sajid@xrmlite special]$ ldd _ufuncs.cpython-36m-x86_64-linux-gnu.so
        linux-vdso.so.1 =>  (0x00007ffefbf84000)
        libmkl_rt.so => /opt/intel/intelpython3/lib/python3.6/site-packages/scipy/special/./../../../../libmkl_rt.so (0x00002aece4e3d000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aece54e9000)
        libpython3.6m.so.1.0 => /opt/intel/intelpython3/lib/python3.6/site-packages/scipy/special/./../../../../libpython3.6m.so.1.0 (0x00002aece5705000)
        libifport.so.5 => /opt/intel/intelpython3/lib/python3.6/site-packages/scipy/special/./../../../../libifport.so.5 (0x00002aece5c42000)
        libifcore.so.5 => /opt/intel/intelpython3/lib/python3.6/site-packages/scipy/special/./../../../../libifcore.so.5 (0x00002aece5e6d000)
        libimf.so => /opt/intel/intelpython3/lib/python3.6/site-packages/scipy/special/./../../../../libimf.so (0x00002aece61c8000)
        libsvml.so => /opt/intel/intelpython3/lib/python3.6/site-packages/scipy/special/./../../../../libsvml.so (0x00002aece6769000)
        libm.so.6 => /lib64/libm.so.6 (0x00002aece810c000)
        libintlc.so.5 => /opt/intel/intelpython3/lib/python3.6/site-packages/scipy/special/./../../../../libintlc.so.5 (0x00002aece840e000)
        libc.so.6 => /lib64/libc.so.6 (0x00002aece867f000)
        libgcc_s.so.1 => /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/gcc-7.3.0-xyzezhjmbiebkjfoakso464rhfshlkyq/lib64/libgcc_s.so.1 (0x00002aece8a42000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00002aece8c59000)
        /lib64/ld-linux-x86-64.so.2 (0x0000560f5adc7000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00002aece8e5e000)
        librt.so.1 => /lib64/librt.so.1 (0x00002aece9061000)

Scipy/Numpy/Python version information:

1.1.0 1.14.3 sys.version_info(major=3, minor=6, micro=6, releaselevel='final', serial=0)

Also, why are both libimf/libsvml and libm linked ? Isn’t the first pair, the intel scalar/vector math library? And if the path to intel libraries is readable, why are only some libraries linked and the others not ?

Update :

Fixed the build flags but now f77 is ifort and f90 is gfortran. Build fails. Build log here.

compiling Fortran sources
Fortran f77 compiler: /home/sajid/packages/spack/lib/spack/env/intel/ifort -FI -fPIC -fp-model strict -O1 -qopenmp 
Fortran f90 compiler: /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/gcc-7.3.0-xyzezhjmbiebkjfoakso464rhfshlkyq/bin/gfortran -FR -fPIC -fp-model strict -O1 -qopenmp 
Fortran fix compiler: /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/gcc-7.3.0-xyzezhjmbiebkjfoakso464rhfshlkyq/bin/gfortran -FI -fPIC -fp-model strict -O1 -qopenmp 

[pv]

It should be also OK to leave F90 unset — in the case above it appears it was explicitly set to gfortran, so at least in hindsight it’s not so surprising that gfortran was called…

[s-sajid-ali]

Works with setting the environment variable F90 to ifort. Thanks for the help!

(Would be great to add directives asking MKL_ROOT to be unset and F90 to be set for intel compilers. )