NVHPC: Unknown Compiler Flags

Hi,

I am trying to compile mpi4py with NVHPC 21.9, which is the follow-up product to the PGI compilers.

Performing a compile with the C/C++ compilers nvc/nvc++ fails with unknown compiler flags being passed to those compilers.

Error Message

...
  running build_clib
  MPI configuration: [mpi] from 'mpi.cfg'
  MPI C compiler:    /opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/mpi/bin/mpicc
  MPI C++ compiler:  /opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/mpi/bin/mpicxx
  MPI F compiler:    /opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/mpi/bin/mpifort
  MPI F90 compiler:  /opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/mpi/bin/mpif90
  MPI F77 compiler:  /opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/mpi/bin/mpif77
  checking for library 'lmpe' ...
  /opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/mpi/bin/mpicc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -c _configtest.c -o _configtest.o
  nvc-Error-Unknown switch: -Wno-unused-result
  nvc-Error-Unknown switch: -fwrapv
  nvc-Error-Unknown switch: -fstack-protector-strong
  nvc-Error-Unknown switch: -fwrapv
  nvc-Error-Unknown switch: -fstack-protector-strong
  nvc-Error-Unknown switch: -Wdate-time
  failure.
...

Reproducer

docker run -it ubuntu:20.04

# Install NVHPC
apt -qqq update
apt install -y \
    build-essential     \
    ca-certificates     \
    cmake               \
    environment-modules \
    gnupg               \
    pkg-config          \
    wget

wget -q https://developer.download.nvidia.com/hpc-sdk/21.9/nvhpc-21-9_21.9_amd64.deb \
        https://developer.download.nvidia.com/hpc-sdk/21.9/nvhpc-2021_21.9_amd64.deb
apt install -y ./nvhpc-21-9_21.9_amd64.deb ./nvhpc-2021_21.9_amd64.deb
rm -rf ./nvhpc-21-9_21.9_amd64.deb ./nvhpc-2021_21.9_amd64.deb
apt install -y python3 python3-pip

source /etc/profile.d/modules.sh
module load /opt/nvidia/hpc_sdk/modulefiles/nvhpc/21.9

# Compile mpi4py
export CC=$(which nvc)
export CXX=$(which nvc++)

python3 -m pip install --upgrade pip setuptools wheel
python3 -m pip install --upgrade -v mpi4py

Extra info:

$ which mpicc 
/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/mpi/bin/mpicc
$ which mpicxx
/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/mpi/bin/mpicxx

$ mpicc --showme
nvc -I/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/openmpi/openmpi-3.1.5/include -Wl,-rpath -Wl,$ORIGIN:$ORIGIN/../../lib:$ORIGIN/../../../lib:$ORIGIN/../../../compilers/lib:$ORIGIN/../../../../compilers/lib:$ORIGIN/../../../../../compilers/lib -Wl,-rpath -Wl,/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/openmpi/openmpi-3.1.5/lib -L/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/openmpi/openmpi-3.1.5/lib -lmpi

$ mpicxx --showme
nvc++ -I/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/openmpi/openmpi-3.1.5/include -Wl,-rpath -Wl,$ORIGIN:$ORIGIN/../../lib:$ORIGIN/../../../lib:$ORIGIN/../../../compilers/lib:$ORIGIN/../../../../compilers/lib:$ORIGIN/../../../../../compilers/lib -Wl,-rpath -Wl,/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/openmpi/openmpi-3.1.5/lib -L/opt/nvidia/hpc_sdk/Linux_x86_64/21.9/comm_libs/openmpi/openmpi-3.1.5/lib -lmpi_cxx -lmpi

CFLAGS and CXXFLAGS are empty.

I cannot spot where those GNU-ish flags come from in the pip/setuptools build logic…

[dalcinl]

The only customization is to replace the compiler command with mpicc, but other than that, no flags are added to build the core MPI extension module (except perhaps on Windows, for the case of MSVC).

I’m working on supporting scikit-build, which uses cmake, to provide an alternative build system. No ETA, though.

[leofang]

@dalcinl @ax3l I suggest us to close this issue given that it’s a unique issue to NVC++ and can be simply worked around. Feel free to open a new issue should there be any action items that I missed.