Cupy cannot be compiled depending on the combination of versions of Setuptools, Numpy, Cython

It seems that Cupy cannot be compiled depending on the combination of versions of Setuptools, Numpy, Cython (and other components).

• Conditions
  • CuPy 9.2.0 (try to compile)
  • Linux OS 5.13.1 / glibc 2.33
  • CUDA 11.4 / NVIDIA driver 470.42.01 (Beta driver which supports CUDA 11.4)
  • Python 3.9.6 / Cython 0.29.33
  • Setuptools 57.1.0 / Numpy 1.21.0
  • GCC 10.2.0 (…also tried using GCC9)
• Error messages that came up when I tried running “DISTUTILS_DEBUG=1 python setup.py build”.

[61/63] Cythonizing cupy_backends/cuda/libs/nvtx.pyx
[62/63] Cythonizing cupy_backends/cuda/libs/profiler.pyx
[63/63] Cythonizing cupy_backends/cuda/stream.pyx
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/numpy/distutils/ccompiler.py", line 735, in new_compiler
    (module_name, class_name, long_description) = compiler_class[compiler]
KeyError: 'nvidia'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/XXX/python-cupy/src/cupy-9.2.0/setup.py", line 145, in <module>
    setup(
  File "/usr/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.9/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.9/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.9/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.9/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/XXX/python-cupy/src/cupy-9.2.0/cupy_setup_build.py", line 1222, in run
    build_ext.build_ext.run(self)
  File "/usr/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/usr/lib/python3.9/site-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/usr/lib/python3.9/distutils/command/build_ext.py", line 306, in run
    self.compiler = new_compiler(compiler=self.compiler,
  File "/usr/lib/python3.9/site-packages/numpy/distutils/ccompiler.py", line 740, in new_compiler
    raise DistutilsPlatformError(msg)
distutils.errors.DistutilsPlatformError: don't know how to compile C/C++ code on platform 'posix' with 'nvidia' compiler

[kmaehashi]

@gdaisukesuzuki Thanks for testing! Glad to know the root cause 😄 We’ll try workarounding it.

[emcastillo]

I reproduced this issue,

just by having pythran installed, cupy can’t be built