question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. ItΒ collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

PyTorch 1.9 Support

See original GitHub issue

Hi,

I have the following library / image versions:

  • PyTorch 1.9.0 with CUDA 11.1
  • Based off this image pytorch/pytorch:1.9.0-cuda11.1-cudnn8-devel

When I try pip install warp_rnnt I have the following output:

Error message
root@a19b333cfe3b:/home/keras/notebook# pip install warp_rnnt
Collecting warp_rnnt
  Using cached warp_rnnt-0.5.0.tar.gz (10 kB)
Requirement already satisfied: pybind11 in /opt/conda/lib/python3.7/site-packages (from warp_rnnt) (2.9.0)
Requirement already satisfied: numpy in /opt/conda/lib/python3.7/site-packages (from warp_rnnt) (1.19.5)
Requirement already satisfied: torch>=1.0.0 in /opt/conda/lib/python3.7/site-packages (from warp_rnnt) (1.9.0)
Requirement already satisfied: typing_extensions in /opt/conda/lib/python3.7/site-packages (from torch>=1.0.0->warp_rnnt) (3.7.4.3)
Building wheels for collected packages: warp-rnnt
  Building wheel for warp-rnnt (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /opt/conda/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/setup.py'"'"'; __file__='"'"'/tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c8
50ea4a028137251/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compil
e(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-73ng5boj
       cwd: /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/
  Complete output (106 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/warp_rnnt
  copying warp_rnnt/__init__.py -> build/lib.linux-x86_64-3.7/warp_rnnt
  copying warp_rnnt/test.py -> build/lib.linux-x86_64-3.7/warp_rnnt
  running build_ext
  building 'warp_rnnt._C' extension
  creating /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7
  Emitting ninja build file /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/build.ninja...
  Compiling objects...
  Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
  [1/3] c++ -MMD -MF /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/binding.o.d -pthread -B /opt/conda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fP
IC -I/opt/conda/lib/python3.7/site-packages/torch/include -I/opt/conda/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.7/site-packages/torch/include/TH -I/opt/conda/lib/python3.7/site-packages/torch/include/THC
-I/usr/local/cuda/include -I/opt/conda/include/python3.7m -c -c /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/binding.cpp -o /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/binding.
o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
  FAILED: /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/binding.o
  c++ -MMD -MF /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/binding.o.d -pthread -B /opt/conda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/
opt/conda/lib/python3.7/site-packages/torch/include -I/opt/conda/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.7/site-packages/torch/include/TH -I/opt/conda/lib/python3.7/site-packages/torch/include/THC -I/usr
/local/cuda/include -I/opt/conda/include/python3.7m -c -c /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/binding.cpp -o /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/binding.o -DTO
RCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
  cc1plus: warning: command line option β€˜-Wstrict-prototypes’ is valid for C/ObjC but not for C++
  In file included from /usr/include/c++/7/ext/string_conversions.h:41:0,
                   from /usr/include/c++/7/bits/basic_string.h:6361,
                   from /usr/include/c++/7/string:52,
                   from /usr/include/c++/7/stdexcept:39,
                   from /usr/include/c++/7/array:39,
                   from /usr/include/c++/7/tuple:39,
                   from /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/binding.cpp:1:
  /usr/include/c++/7/cstdlib:75:15: fatal error: stdlib.h: No such file or directory
   #include_next <stdlib.h>
                 ^~~~~~~~~~                                                                                                                                                                                                                                    compilation terminated.                                                                                        
  compilation terminated.                                                                                                                                                                                                                                      [2/3] /usr/local/cuda/bin/nvcc  -I/opt/conda/lib/python3.7/site-packages/torch/include -I/opt/conda/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.7/site-packages/torch/include/TH -I/opt/conda/lib/python3.7/site-packages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/include/python3.7m -c -c /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/core.cu -o /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/tem
p.linux-x86_64-3.7/core.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIN
D11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,co
de=sm_86 -std=c++14
  FAILED: /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/core.o
  /usr/local/cuda/bin/nvcc  -I/opt/conda/lib/python3.7/site-packages/torch/include -I/opt/conda/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.7/site-packages/torch/include/TH -I/opt/conda/lib/python3.7/site-pa
ckages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/include/python3.7m -c -c /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/core.cu -o /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linu
x-x86_64-3.7/core.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_CO
MPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_
86 -std=c++14
  In file included from /usr/local/cuda/include/crt/math_functions.h:8958:0,
                   from /usr/local/cuda/include/crt/common_functions.h:295,
                   from /usr/local/cuda/include/cuda_runtime.h:115,
                   from <command-line>:0:
  /usr/include/c++/7/cmath:45:15: fatal error: math.h: No such file or directory
   #include_next <math.h>
                 ^~~~~~~~
  compilation terminated.
  [3/3] /usr/local/cuda/bin/nvcc  -I/opt/conda/lib/python3.7/site-packages/torch/include -I/opt/conda/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.7/site-packages/torch/include/TH -I/opt/conda/lib/python3.7/s
ite-packages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/include/python3.7m -c -c /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/core_gather.cu -o /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/bu
ild/temp.linux-x86_64-3.7/core_gather.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENS
ION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_86,code=compute_86 -gencode=arch
=compute_86,code=sm_86 -std=c++14
  FAILED: /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/core_gather.o
  /usr/local/cuda/bin/nvcc  -I/opt/conda/lib/python3.7/site-packages/torch/include -I/opt/conda/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.7/site-packages/torch/include/TH -I/opt/conda/lib/python3.7/site-pa
ckages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/include/python3.7m -c -c /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/core_gather.cu -o /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/te
mp.linux-x86_64-3.7/core_gather.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H
'-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compu
te_86,code=sm_86 -std=c++14
  In file included from /usr/local/cuda/include/crt/math_functions.h:8958:0,
                   from /usr/local/cuda/include/crt/common_functions.h:295,
                   from /usr/local/cuda/include/cuda_runtime.h:115,
                   from <command-line>:0:
  /usr/include/c++/7/cmath:45:15: fatal error: math.h: No such file or directory
   #include_next <math.h>
                 ^~~~~~~~
  compilation terminated.
  ninja: build stopped: subcommand failed.  

Traceback (most recent call last): File β€œ/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py”, line 1672, in _run_ninja_build env=env) File β€œ/opt/conda/lib/python3.7/subprocess.py”, line 512, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command β€˜[β€˜ninja’, β€˜-v’]’ returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File β€œ<string>”, line 1, in <module> File β€œ/tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/setup.py”, line 64, in <module> β€œTopic :: Software Development :: Libraries :: Python Modules”, File β€œ/opt/conda/lib/python3.7/site-packages/setuptools/init.py”, line 153, in setup return distutils.core.setup(**attrs) File β€œ/opt/conda/lib/python3.7/distutils/core.py”, line 148, in setup dist.run_commands() File β€œ/opt/conda/lib/python3.7/distutils/dist.py”, line 966, in run_commands self.run_command(cmd) File β€œ/opt/conda/lib/python3.7/distutils/dist.py”, line 985, in run_command cmd_obj.run() File β€œ/opt/conda/lib/python3.7/site-packages/wheel/bdist_wheel.py”, line 290, in run self.run_command(β€˜build’) File β€œ/opt/conda/lib/python3.7/distutils/cmd.py”, line 313, in run_command self.distribution.run_command(command) File β€œ/opt/conda/lib/python3.7/distutils/dist.py”, line 985, in run_command cmd_obj.run() File β€œ/opt/conda/lib/python3.7/site-packages/setuptools/command/build_ext.py”, line 79, in run _build_ext.run(self) File β€œ/opt/conda/lib/python3.7/distutils/command/build_ext.py”, line 340, in run self.build_extensions() File β€œ/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py”, line 709, in build_extensions build_ext.build_extensions(self) File β€œ/opt/conda/lib/python3.7/distutils/command/build_ext.py”, line 449, in build_extensions self._build_extensions_serial() File β€œ/opt/conda/lib/python3.7/distutils/command/build_ext.py”, line 474, in _build_extensions_serial self.build_extension(ext) File β€œ/opt/conda/lib/python3.7/site-packages/setuptools/command/build_ext.py”, line 196, in build_extension _build_ext.build_extension(self, ext) File β€œ/opt/conda/lib/python3.7/distutils/command/build_ext.py”, line 534, in build_extension depends=ext.depends) File β€œ/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py”, line 539, in unix_wrap_ninja_compile with_cuda=with_cuda) File β€œ/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py”, line 1360, in _write_ninja_file_and_compile_objects error_prefix=β€˜Error compiling objects for extension’) File β€œ/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py”, line 1682, in _run_ninja_build raise RuntimeError(message) from e RuntimeError: Error compiling objects for extension

ERROR: Failed building wheel for warp-rnnt Running setup.py clean for warp-rnnt Failed to build warp-rnnt Installing collected packages: warp-rnnt Running setup.py install for warp-rnnt … error ERROR: Command errored out with exit status 1: command: /opt/conda/bin/python -u -c β€˜import io, os, sys, setuptools, tokenize; sys.argv[0] = β€˜"’"’/tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/setup.pyβ€™β€œ'β€β€˜; file=β€™β€œ'β€β€˜/tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427 c850ea4a028137251/setup.pyβ€™β€œ'β€β€˜;f = getattr(tokenize, β€˜"β€™β€œβ€˜openβ€™β€β€™β€œβ€˜, open)(file) if os.path.exists(file) else io.StringIO(β€™β€β€˜β€œβ€˜from setuptools import setup; setup()β€™β€β€™β€œβ€˜);code = f.read().replace(β€™β€β€˜"’\r\nβ€™β€œ'β€β€˜, β€˜"’"’\nβ€™β€œ'β€β€˜);f.close();exec(comp ile(code, file, β€˜"β€™β€œβ€˜execβ€™β€β€™β€œβ€˜))’ install --record /tmp/pip-record-vvd1smyc/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/include/python3.7m/warp-rnnt cwd: /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/ Complete output (108 lines): running install running build running build_py creating build creating build/lib.linux-x86_64-3.7 creating build/lib.linux-x86_64-3.7/warp_rnnt copying warp_rnnt/init.py -> build/lib.linux-x86_64-3.7/warp_rnnt copying warp_rnnt/test.py -> build/lib.linux-x86_64-3.7/warp_rnnt running build_ext building 'warp_rnnt.C’ extension creating /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7 Emitting ninja build file /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/build.ninja… Compiling objects… Allowing ninja to set a default number of workers… (overridable by setting the environment variable MAX_JOBS=N) [1/3] c++ -MMD -MF /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/binding.o.d -pthread -B /opt/conda/compiler_compat -Wl,–sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes - fPIC -I/opt/conda/lib/python3.7/site-packages/torch/include -I/opt/conda/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.7/site-packages/torch/include/TH -I/opt/conda/lib/python3.7/site-packages/torch/include/TH C -I/usr/local/cuda/include -I/opt/conda/include/python3.7m -c -c /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/binding.cpp -o /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/bindin g.o -DTORCH_API_INCLUDE_EXTENSION_H β€˜-DPYBIND11_COMPILER_TYPE=β€œgcc"’ '-DPYBIND11_STDLIB="libstdcpp"’ '-DPYBIND11_BUILD_ABI="cxxabi1011"’ -DTORCH_EXTENSION_NAME=C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14 FAILED: /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/binding.o c++ -MMD -MF /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/binding.o.d -pthread -B /opt/conda/compiler_compat -Wl,–sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC - I/opt/conda/lib/python3.7/site-packages/torch/include -I/opt/conda/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.7/site-packages/torch/include/TH -I/opt/conda/lib/python3.7/site-packages/torch/include/THC -I/u sr/local/cuda/include -I/opt/conda/include/python3.7m -c -c /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/binding.cpp -o /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/binding.o -D TORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="gcc"’ '-DPYBIND11_STDLIB="libstdcpp"’ '-DPYBIND11_BUILD_ABI="cxxabi1011"’ -DTORCH_EXTENSION_NAME=C -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14 cc1plus: warning: command line option β€˜-Wstrict-prototypes’ is valid for C/ObjC but not for C++ In file included from /usr/include/c++/7/ext/string_conversions.h:41:0, from /usr/include/c++/7/bits/basic_string.h:6361, from /usr/include/c++/7/string:52, from /usr/include/c++/7/stdexcept:39, from /usr/include/c++/7/array:39, from /usr/include/c++/7/tuple:39, from /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/binding.cpp:1: /usr/include/c++/7/cstdlib:75:15: fatal error: stdlib.h: No such file or directory #include_next <stdlib.h> ^~~~~~~~~~ compilation terminated. [2/3] /usr/local/cuda/bin/nvcc -I/opt/conda/lib/python3.7/site-packages/torch/include -I/opt/conda/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.7/site-packages/torch/include/TH -I/opt/conda/lib/python3.7 /site-packages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/include/python3.7m -c -c /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/core_gather.cu -o /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/ build/temp.linux-x86_64-3.7/core_gather.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options β€˜β€™β€β€™β€œβ€˜-fPICβ€™β€β€˜"’’ -DTORCH_API_INCLUDE_EXTE NSION_H '-DPYBIND11_COMPILER_TYPE="gcc"’ '-DPYBIND11_STDLIB="libstdcpp"’ '-DPYBIND11_BUILD_ABI="cxxabi1011"’ -DTORCH_EXTENSION_NAME=C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_86,code=compute_86 -gencode=ar ch=compute_86,code=sm_86 -std=c++14 FAILED: /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/core_gather.o /usr/local/cuda/bin/nvcc -I/opt/conda/lib/python3.7/site-packages/torch/include -I/opt/conda/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.7/site-packages/torch/include/TH -I/opt/conda/lib/python3.7/site- packages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/include/python3.7m -c -c /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/core_gather.cu -o /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/ temp.linux-x86_64-3.7/core_gather.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS_ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options β€˜β€™β€β€˜"’-fPICβ€™β€œ'β€β€˜β€™ -DTORCH_API_INCLUDE_EXTENSION_ H β€˜-DPYBIND11_COMPILER_TYPE=β€œgcc"’ '-DPYBIND11_STDLIB="libstdcpp"’ '-DPYBIND11_BUILD_ABI="cxxabi1011"’ -DTORCH_EXTENSION_NAME=C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_86,code=compute_86 -gencode=arch=com pute_86,code=sm_86 -std=c++14 In file included from /usr/local/cuda/include/crt/math_functions.h:8958:0, from /usr/local/cuda/include/crt/common_functions.h:295, from /usr/local/cuda/include/cuda_runtime.h:115, from <command-line>:0: /usr/include/c++/7/cmath:45:15: fatal error: math.h: No such file or directory #include_next <math.h> ^~~~~~~~ compilation terminated. [3/3] /usr/local/cuda/bin/nvcc -I/opt/conda/lib/python3.7/site-packages/torch/include -I/opt/conda/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.7/site-packages/torch/include/TH -I/opt/conda/lib/python3.7 /site-packages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/include/python3.7m -c -c /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/core.cu -o /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/t emp.linux-x86_64-3.7/core.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options β€˜β€™β€β€™β€œβ€˜-fPICβ€™β€β€˜"’’ -DTORCH_API_INCLUDE_EXTENSION_H β€˜-DPYB IND11_COMPILER_TYPE=β€œgcc"’ '-DPYBIND11_STDLIB="libstdcpp"’ '-DPYBIND11_BUILD_ABI="cxxabi1011"’ -DTORCH_EXTENSION_NAME=C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86, code=sm_86 -std=c++14 FAILED: /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.linux-x86_64-3.7/core.o /usr/local/cuda/bin/nvcc -I/opt/conda/lib/python3.7/site-packages/torch/include -I/opt/conda/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/opt/conda/lib/python3.7/site-packages/torch/include/TH -I/opt/conda/lib/python3.7/site- packages/torch/include/THC -I/usr/local/cuda/include -I/opt/conda/include/python3.7m -c -c /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/core.cu -o /tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/build/temp.li nux-x86_64-3.7/core.o -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options β€˜β€™β€β€™β€œβ€˜-fPICβ€™β€β€˜"’’ -DTORCH_API_INCLUDE_EXTENSION_H β€˜-DPYBIND11_ COMPILER_TYPE=β€œ_gcc”’ β€˜-DPYBIND11_STDLIB=β€œ_libstdcpp”’ β€˜-DPYBIND11_BUILD_ABI=β€œ_cxxabi1011”’ -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=s m_86 -std=c++14 In file included from /usr/local/cuda/include/crt/math_functions.h:8958:0, from /usr/local/cuda/include/crt/common_functions.h:295, from /usr/local/cuda/include/cuda_runtime.h:115, from <command-line>:0: /usr/include/c++/7/cmath:45:15: fatal error: math.h: No such file or directory #include_next <math.h> ^~~~~~~~ compilation terminated. ninja: build stopped: subcommand failed. Traceback (most recent call last): File β€œ/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py”, line 1672, in _run_ninja_build env=env) File β€œ/opt/conda/lib/python3.7/subprocess.py”, line 512, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command β€˜[β€˜ninja’, β€˜-v’]’ returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/setup.py", line 64, in <module>
    "Topic :: Software Development :: Libraries :: Python Modules",
  File "/opt/conda/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/opt/conda/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/opt/conda/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/opt/conda/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run
    return orig.install.run(self)
  File "/opt/conda/lib/python3.7/distutils/command/install.py", line 545, in run
    self.run_command('build')
  File "/opt/conda/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.7/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/opt/conda/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 79, in run
    _build_ext.run(self)
  File "/opt/conda/lib/python3.7/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 709, in build_extensions
    build_ext.build_extensions(self)
  File "/opt/conda/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/opt/conda/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/opt/conda/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
    _build_ext.build_extension(self, ext)
  File "/opt/conda/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
    depends=ext.depends)
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 539, in unix_wrap_ninja_compile
    with_cuda=with_cuda)
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1360, in _write_ninja_file_and_compile_objects
    error_prefix='Error compiling objects for extension')
  File "/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1682, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension
----------------------------------------

ERROR: Command errored out with exit status 1: /opt/conda/bin/python -u -c β€˜import io, os, sys, setuptools, tokenize; sys.argv[0] = β€˜"’"’/tmp/pip-install-b1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/setup.pyβ€™β€œ'β€β€˜; file=β€™β€œ'β€β€˜/tmp/pip-install-b 1mhbvnc/warp-rnnt_074f8d6e4c20427c850ea4a028137251/setup.pyβ€™β€œ'β€β€˜;f = getattr(tokenize, β€˜"β€™β€œβ€˜openβ€™β€β€™β€œβ€˜, open)(file) if os.path.exists(file) else io.StringIO(β€™β€β€˜β€œβ€˜from setuptools import setup; setup()β€™β€β€™β€œβ€˜);code = f.read().replace(β€™β€β€˜"’\r\nβ€™β€œ'β€β€˜, β€˜"’"’\nβ€™β€œ'β€β€˜);f.close();exec(compile(code, file, β€˜"β€™β€œβ€˜exec’”’"β€˜))’ install --record /tmp/pip-record-vvd1smyc/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/include/python3.7m/warp-rnnt Check the logs fo r full command output.

I used to have similar errors with warp-ctc, and they were resolved with hacks like this:

RUN apt-get install gcc-5 g++-5 g++-5-multilib gfortran-5 -y && \
    update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 60 --slave /usr/bin/g++ g++ /usr/bin/g++-5 --slave /usr/bin/gfortran gfortran /usr/bin/gfortran-5 && \
    update-alternatives --query gcc
RUN gcc --version

Is there anything I can do about this without downgrading or changing the current environment?

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
1yticcommented, Jan 5, 2022

I checked this version pip3 install torch==1.9.0+cu111 -f https://download.pytorch.org/whl/cu111/torch_stable.html and warp_rnnt package compiled successfully. I also have the local cuda install with the same version 11.1.

0reactions
snakers4commented, Jan 5, 2022

I see. I tried using these images from scratch:

docker pull pytorch/pytorch:1.10.0-cuda11.3-cudnn8-devel
docker pull pytorch/pytorch:1.9.0-cuda11.1-cudnn8-devel

And it compiled successfully. Looks like the problem is with our images, we most likely borked something up inside of them. Many thanks.

Read more comments on GitHub >

github_iconTop Results From Across the Web

PyTorch 1.9 Release, including torch.linalg and Mobile ...
Major improvements to support scientific computing, including torch.linalg, torch.special, and Complex Autograd; Major improvements in on-deviceΒ ...
Read more >
Releases Β· pytorch/pytorch - GitHub
PyTorch 1.13: beta versions of functorch and improved support for Apple's new M1 ... This release is composed of over 3,400 commits since...
Read more >
How does one install pytorch 1.9 in an HPC that seems to ...
First of all, as @Francois suggested, try to uninstall the CPU only version of pytorch. Also in your installation script, you should useΒ ......
Read more >
PyTorch Release 21.04 - NVIDIA Documentation Center
PyTorch container image version 21.04 is based on 1.9.0a0+2ecb2c7 ... CUDA, PyTorch, and TensorRT are supported in each of the NVIDIAΒ ...
Read more >
torch - PyPI
PyTorch is a Python package that provides two high-level features: ... torch, A Tensor library like NumPy, with strong GPU support.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found