Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BUG: Build fails with latest intel oneapi

See original GitHub issue

Describe your issue.

I am trying to build scipy in a singularity image based on the latest intel oneapi release, but the compilation fails

Reproducing Code Example

#Complete singularity script to be executed with sudo singularity build image.sif [content_in_file].def

Bootstrap: docker
From: ubuntu:focal

%setup
    mkdir -p /root/.ssh/

%files
    /home/$USER/.ssh/id_rsa /root/.ssh/id_rsa
    /home/$USER/.ssh/known_hosts /root/.ssh/known_hosts

%environment
    export LC_ALL=C

%post
    cd /root
    chmod 600 .ssh/id_rsa

    export DEBIAN_FRONTEND=noninteractive
    apt-get -y update
    apt-get -y install wget libssl-dev make cmake gcc git vim libeigen3-dev python3-pip libboost-all-dev libhdf5-dev libxml2-dev libpciaccess-dev python3-pybind11 software-properties-common
    apt-get -y update
    pip3 install --upgrade pip

    wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB -O - | apt-key add -
    echo "deb https://apt.repos.intel.com/oneapi all main" | tee /etc/apt/sources.list.d/oneAPI.list
    add-apt-repository "deb https://apt.repos.intel.com/oneapi all main"
    apt-get -y update
    apt-get -y install intel-hpckit

    export CC=/opt/intel/oneapi/compiler/2021.4.0/linux/bin/intel64/icc 
export   CXX=/opt/intel/oneapi/compiler/2021.4.0/linux/bin/intel64/icpc 
export   FC=/opt/intel/oneapi/compiler/2021.4.0/linux/bin/intel64/ifort 
export   F77=/opt/intel/oneapi/compiler/2021.4.0/linux/bin/intel64/ifort 
export   F90=/opt/intel/oneapi/compiler/2021.4.0/linux/bin/intel64/ifort 
export   MKLROOT=/opt/intel/oneapi/mkl/2021.4.0 
export   TBBROOT=/opt/intel/oneapi/tbb/2021.4.0/env/.. 
export   CPLUS_INCLUDE_PATH=/opt/intel/oneapi/clck/2021.4.0/include 
export  INFOPATH=/opt/intel/oneapi/debugger/10.2.4/gdb/intel64/lib:$INFOPATH
export  LIBRARY_PATH=/opt/intel/oneapi/vpl/2021.6.0/lib:/opt/intel/oneapi/tbb/2021.4.0/env/../lib/intel64/gcc4.8:/opt/intel/oneapi/mpi/2021.4.0//libfabric/lib:/opt/intel/oneapi/mpi/2021.4.0//lib/release:/opt/intel/oneapi/mpi/2021.4.0//lib:/opt/intel/oneapi/mkl/2021.4.0/lib/intel64:/opt/intel/oneapi/ipp/2021.4.0/lib/intel64:/opt/intel/oneapi/ippcp/2021.4.0/lib/intel64:/opt/intel/oneapi/ipp/2021.4.0/lib/intel64:/opt/intel/oneapi/dnnl/2021.4.0/cpu_dpcpp_gpu_dpcpp/lib:/opt/intel/oneapi/dal/2021.4.0/lib/intel64:/opt/intel/oneapi/compiler/2021.4.0/linux/compiler/lib/intel64_lin:/opt/intel/oneapi/compiler/2021.4.0/linux/lib:/opt/intel/oneapi/clck/2021.4.0/lib/intel64:/opt/intel/oneapi/ccl/2021.4.0/lib/cpu_gpu_dpcpp:$LIBRARY_PATH
export  LD_LIBRARY_PATH=/opt/intel/oneapi/vpl/2021.6.0/lib:/opt/intel/oneapi/tbb/2021.4.0/env/../lib/intel64/gcc4.8:/opt/intel/oneapi/mpi/2021.4.0//libfabric/lib:/opt/intel/oneapi/mpi/2021.4.0//lib/release:/opt/intel/oneapi/mpi/2021.4.0//lib:/opt/intel/oneapi/mkl/2021.4.0/lib/intel64:/opt/intel/oneapi/itac/2021.4.0/slib:/opt/intel/oneapi/ipp/2021.4.0/lib/intel64:/opt/intel/oneapi/ippcp/2021.4.0/lib/intel64:/opt/intel/oneapi/ipp/2021.4.0/lib/intel64:/opt/intel/oneapi/dnnl/2021.4.0/cpu_dpcpp_gpu_dpcpp/lib:/opt/intel/oneapi/debugger/10.2.4/gdb/intel64/lib:/opt/intel/oneapi/debugger/10.2.4/libipt/intel64/lib:/opt/intel/oneapi/debugger/10.2.4/dep/lib:/opt/intel/oneapi/dal/2021.4.0/lib/intel64:/opt/intel/oneapi/compiler/2021.4.0/linux/lib:/opt/intel/oneapi/compiler/2021.4.0/linux/lib/x64:/opt/intel/oneapi/compiler/2021.4.0/linux/lib/emu:/opt/intel/oneapi/compiler/2021.4.0/linux/lib/oclfpga/host/linux64/lib:/opt/intel/oneapi/compiler/2021.4.0/linux/lib/oclfpga/linux64/lib:/opt/intel/oneapi/compiler/2021.4.0/linux/compiler/lib/intel64_lin:/opt/intel/oneapi/ccl/2021.4.0/lib/cpu_gpu_dpcpp:$LD_LIBRARY_PATH
export  PATH=/opt/intel/oneapi/vtune/2021.7.1/bin64:/opt/intel/oneapi/vpl/2021.6.0/bin:/opt/intel/oneapi/mpi/2021.4.0//libfabric/bin:/opt/intel/oneapi/mpi/2021.4.0//bin:/opt/intel/oneapi/mkl/2021.4.0/bin/intel64:/opt/intel/oneapi/itac/2021.4.0/bin:/opt/intel/oneapi/intelpython/latest/bin:/opt/intel/oneapi/intelpython/latest/condabin:/opt/intel/oneapi/inspector/2021.4.0/bin64:/opt/intel/oneapi/dpcpp-ct/2021.4.0/bin:/opt/intel/oneapi/dev-utilities/2021.4.0/bin:/opt/intel/oneapi/debugger/10.2.4/gdb/intel64/bin:/opt/intel/oneapi/compiler/2021.4.0/linux/lib/oclfpga/llvm/aocl-bin:/opt/intel/oneapi/compiler/2021.4.0/linux/lib/oclfpga/bin:/opt/intel/oneapi/compiler/2021.4.0/linux/bin/intel64:/opt/intel/oneapi/compiler/2021.4.0/linux/bin:/opt/intel/oneapi/clck/2021.4.0/bin/intel64:/opt/intel/oneapi/advisor/2021.4.0/bin64:/home/cds/tweymuth/bin:/home/cds/tweymuth/bin:/data/soft/centos7/latex/texlive/2016/bin/x86_64-linux:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/data/soft/centos7/utils/print:/data/soft/centos7/utils/nodeload:$PATH
export  GDB_INFO=/opt/intel/oneapi/debugger/10.2.4/documentation/info/:$GDB_INFO
export  CMAKE_PREFIX_PATH=/opt/intel/oneapi/vpl/2021.6.0:/opt/intel/oneapi/tbb/2021.4.0/env/..:/opt/intel/oneapi/dal/2021.4.0:$CMAKE_PREFIX_PATH
export   CONDA_SHLVL=1 
export   CONDA_PYTHON_EXE=/opt/intel/oneapi/intelpython/latest/bin/python 
export  PYTHONPATH=/opt/intel/oneapi/advisor/2021.4.0/pythonapi:$PYTHONPATH
export   CONDA_DEFAULT_ENV=intelpython-python3.7 
export   INTEL_PYTHONHOME=/opt/intel/oneapi/debugger/10.2.4/dep 
export   CONDA_EXE=/opt/intel/oneapi/intelpython/latest/bin/conda 
export   CONDA_PREFIX=/opt/intel/oneapi/intelpython/latest 
export   VT_MPI=impi4 
export   IPPROOT=/opt/intel/oneapi/ipp/2021.4.0 
export   INTEL_LICENSE_FILE=/opt/intel/oneapi/clck/2021.4.0/licensing 
export   CLCK_ROOT=/opt/intel/oneapi/clck/2021.4.0 
export   OCL_ICD_FILENAMES=libintelocl_emu.so:libalteracl.so:/opt/intel/oneapi/compiler/2021.4.0/linux/lib/x64/libintelocl.so 
export   CCL_CONFIGURATION=cpu_gpu_dpcpp 
export   IPPCRYPTOROOT=/opt/intel/oneapi/ippcp/2021.4.0 
export   FPGA_VARS_DIR=/opt/intel/oneapi/compiler/2021.4.0/linux/lib/oclfpga 
export   FI_PROVIDER_PATH=/opt/intel/oneapi/mpi/2021.4.0//libfabric/lib/prov:/usr/lib64/libfabric 
export   CMPLR_ROOT=/opt/intel/oneapi/compiler/2021.4.0 
export  CPATH=/opt/intel/oneapi/vpl/2021.6.0/include:/opt/intel/oneapi/tbb/2021.4.0/env/../include:/opt/intel/oneapi/mpi/2021.4.0//include:/opt/intel/oneapi/mkl/2021.4.0/include:/opt/intel/oneapi/ipp/2021.4.0/include:/opt/intel/oneapi/ippcp/2021.4.0/include:/opt/intel/oneapi/ipp/2021.4.0/include:/opt/intel/oneapi/dpl/2021.5.0/linux/include:/opt/intel/oneapi/dpcpp-ct/2021.4.0/include:/opt/intel/oneapi/dnnl/2021.4.0/cpu_dpcpp_gpu_dpcpp/lib:/opt/intel/oneapi/dev-utilities/2021.4.0/include:/opt/intel/oneapi/dal/2021.4.0/include:/opt/intel/oneapi/compiler/2021.4.0/linux/include:/opt/intel/oneapi/ccl/2021.4.0/include/cpu_gpu_dpcpp:$CPATH
export   DPL_ROOT=/opt/intel/oneapi/dpl/2021.5.0 
export   INTELFPGAOCLSDKROOT=/opt/intel/oneapi/compiler/2021.4.0/linux/lib/oclfpga 
export   IPPCP_TARGET_ARCH=intel64 
export   SETVARS_COMPLETED=1 
export   NLSPATH=/opt/intel/oneapi/mkl/2021.4.0/lib/intel64/locale/%l_%t/%N 
export   VT_ADD_LIBS="-ldwarf -lelf -lvtunwind -lm -lpthread"
export   DPCT_BUNDLE_ROOT=/opt/intel/oneapi/dpcpp-ct/2021.4.0 
export   DNNLROOT=/opt/intel/oneapi/dnnl/2021.4.0/cpu_dpcpp_gpu_dpcpp 
export   VT_LIB_DIR=/opt/intel/oneapi/itac/2021.4.0/lib 
export   INSPECTOR_2021_DIR=/opt/intel/oneapi/inspector/2021.4.0 
export   DAALROOT=/opt/intel/oneapi/dal/2021.4.0 
export   DALROOT=/opt/intel/oneapi/dal/2021.4.0 
export   DAL_MAJOR_BINARY=1 
export   DAL_MINOR_BINARY=1 
export   VT_ROOT=/opt/intel/oneapi/itac/2021.4.0 
export   IPP_TARGET_ARCH=intel64 
export   CCL_ROOT=/opt/intel/oneapi/ccl/2021.4.0 
export   VT_SLIB_DIR=/opt/intel/oneapi/itac/2021.4.0/slib 
export   VTUNE_PROFILER_2021_DIR=/opt/intel/oneapi/vtune/2021.7.1 
export   ACL_BOARD_VENDOR_PATH=/opt/Intel/OpenCLFPGA/oneAPI/Boards 
export   APM=/opt/intel/oneapi/advisor/2021.4.0/perfmodels 
export  CLASSPATH=/opt/intel/oneapi/mpi/2021.4.0//lib/mpi.jar:/opt/intel/oneapi/dal/2021.4.0/lib/onedal.jar:$CLASSPATH
export  PKG_CONFIG_PATH=/opt/intel/oneapi/vtune/2021.7.1/include/pkgconfig/lib64:/opt/intel/oneapi/vpl/2021.6.0/lib/pkgconfig:/opt/intel/oneapi/tbb/2021.4.0/env/../lib/pkgconfig:/opt/intel/oneapi/mpi/2021.4.0/lib/pkgconfig:/opt/intel/oneapi/mkl/2021.4.0/lib/pkgconfig:/opt/intel/oneapi/ippcp/2021.4.0/lib/pkgconfig:/opt/intel/oneapi/inspector/2021.4.0/include/pkgconfig/lib64:/opt/intel/oneapi/dpl/2021.5.0/lib/pkgconfig:/opt/intel/oneapi/dal/2021.4.0/lib/pkgconfig:/opt/intel/oneapi/compiler/2021.4.0/lib/pkgconfig:/opt/intel/oneapi/ccl/2021.4.0/lib/pkgconfig:/opt/intel/oneapi/advisor/2021.4.0/include/pkgconfig/lib64:$PKG_CONFIG_PATH
export   ADVISOR_2021_DIR=/opt/intel/oneapi/advisor/2021.4.0 
export   SETVARS_VARS_PATH=/opt/intel/oneapi/vtune/latest/env/vars.sh 
export   ONEAPI_ROOT=/opt/intel/oneapi 
export   I_MPI_ROOT=/opt/intel/oneapi/mpi/2021.4.0 
    ln -r -s /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/libmkl_avx2.so.1 /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/libmkl_avx2.so
    ln -r -s /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/libmkl_vml_avx2.so.1 /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/libmkl_vml_avx2.so
    echo 'export LD_LIBRARY_PATH=/opt/intel/oneapi/mkl/2021.4.0/lib/intel64:$LD_LIBRARY_PATH' >>$SINGULARITY_ENVIRONMENT
    echo 'export MKLROOT=/opt/intel/oneapi/mkl/2021.4.0' >>$SINGULARITY_ENVIRONMENT

    pip3 install pybind11
    pip3 install Cython pythran

    # Install numpy
    git clone https://github.com/numpy/numpy.git numpy
    cd numpy
    git checkout v1.21.2
    cp site.cfg.example site.cfg
    echo "\n[mkl]" >> site.cfg
    echo "include_dirs = /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/" >> site.cfg
    echo "library_dirs = /opt/intel/oneapi/mkl/2021.4.0/lib/intel64/" >> site.cfg
    echo "mkl_libs = mkl_rt" >> site.cfg
    echo "lapack_libs =" >> site.cfg
    python3 setup.py build --fcompiler=intelem --compiler=intelem
    python3 setup.py install
    cd ..
    rm -rf numpy

    # Install scipy
    git clone https://github.com/scipy/scipy.git scipy
    cd scipy
    git checkout v1.7.1
    git submodule update --init
    # next line fails
    python3 setup.py build --fcompiler=intelem --compiler=intelem
    python3 setup.py install
    cd ..
    rm -rf scipy



# but simply the last code block also fails for my normal centos7 system with the intel compiler set-up with the identical error
    git clone https://github.com/scipy/scipy.git scipy
    cd scipy
    git checkout v1.7.1
    git submodule update --init
    # next line fails
    python3 setup.py build --fcompiler=intelem --compiler=intelem

Error message

/opt/intel/oneapi/intelpython/latest/lib/python3.7/site-packages/pythran/pythonic/include/types/numpy_gexpr.hpp(331): internal error: assertion failed at: "expr.c", line 9750 in symbol_for_template_param_unknown_entity_rescan

                std::get<0>(t1) * std::get<0>(t0).step + std::get<0>(t0).lower,
                                  ^

compilation aborted for scipy/interpolate/_rbfinterp_pythran.cpp (code 4)
error: Command "icc -m64 -fPIC -fp-model strict -O3 -fomit-frame-pointer -xSSE4.2 -axCORE-AVX2,COMMON-AVX512 -DENABLE_PYTHON_MODULE -D__PYTHRAN__=3 -DPYTHRAN_BLAS_NONE -I/opt/intel/oneapi/intelpython/latest/lib/python3.7/site-packages/pythran -I/opt/intel/oneapi/intelpython/latest/lib/python3.7/site-packages/numpy/core/include -I/opt/intel/oneapi/intelpython/latest/lib/python3.7/site-packages/numpy/core/include -Ibuild/src.linux-x86_64-3.7/numpy/distutils/include -I/opt/intel/oneapi/intelpython/latest/include/python3.7m -c scipy/interpolate/_rbfinterp_pythran.cpp -o build/temp.linux-x86_64-3.7/scipy/interpolate/_rbfinterp_pythran.o -MMD -MF build/temp.linux-x86_64-3.7/scipy/interpolate/_rbfinterp_pythran.o.d -std=c++11 -fno-math-errno -fvisibility=hidden -fno-wrapv -Wno-unused-function -Wno-int-in-bool-context -Wno-unknown-warning-option -msse -msse2 -msse3" failed with exit status 4

SciPy/NumPy/Python version information

1.3.2 1.17.4 sys.version_info(major=3, minor=7, micro=6, releaselevel=‘final’, serial=0)

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:18 (12 by maintainers)

github_iconTop GitHub Comments

1reaction
boegelcommented, Jul 7, 2022

I tested this by installing scipy (v1.8.1) on top of pythran commit https://github.com/serge-sans-paille/pythran/commit/eb52ee93cb78d2afba50cc9418ac6bf9567fb958 without using export SCIPY_USE_PYTHRAN=0 first, using the latest (classic) Intel compilers 2022.1, and that worked without trouble.

0reactions
rgommerscommented, Jul 7, 2022

Awesome, thanks for confirming. I’ll close this then. It will require the next Pythran release, which will be 0.11.1 or 0.12.0.

Read more comments on GitHub >

github_iconTop Results From Across the Web

[BUG] build fails with IntelLLVM 2022.2.x compiler · Issue #1505
Attempting to build google benchmark fails with the latest intel compiler in release mode The OS is ubuntu 22.04, Intel LLVM compiler 2022.2 ......
Read more >
Error building OpenMPI with intel compiler: - Intel Communities
I am trying to build openmpi library that works with this intel oneAPI. In past I have built separate openmpi libraries that go...
Read more >
Intel oneAPI Base Toolkit installation fails with error
The installation of the Intel oneAPI Base Toolkit version 2022.1 application may fail with an error such as the following:
Read more >
Error in compiling nwchem-7.0.2 with Intel oneapi
bash script to compile NWChem with intel compilers and MKL ... export MPI_LOC=/home/lib/intel/oneapi/mpi/latest export MPI_LIB=$MPI_LOC/lib
Read more >
How can I fix the compling error on oneAPI code
The issue might be with the version of GCC that you have in your system ... the below link, intel.com/content/www/us/en/developer/articles/…
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

BUG: orthogonal_procrustes returns the transpose/inverse

Next page

CI job is choking on new NumPy memory allocator warnings