cupy.cuda.compiler.CompileException: nvrtc: error: failed to load builtins

When you run the demo: python -m cmr.demo --name bird_net --num_train_epoch 500 --img_path cmr/demo_data/img1.jpg

I am getting this error:

/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters Setting up model… loading /home/ubuntu/akshay/cmr/nnutils/…/cachedir/snapshots/bird_net/pred_net_500.pth… Traceback (most recent call last): File “/usr/lib/python2.7/runpy.py”, line 174, in _run_module_as_main “main”, fname, loader, pkg_name) File “/usr/lib/python2.7/runpy.py”, line 72, in _run_code exec code in run_globals File “/home/ubuntu/akshay/cmr/demo.py”, line 119, in <module> app.run(main) File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/absl/app.py”, line 274, in run _run_main(main, argv) File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/absl/app.py”, line 238, in _run_main sys.exit(main(argv)) File “/home/ubuntu/akshay/cmr/demo.py”, line 108, in main outputs = predictor.predict(batch) File “cmr/nnutils/predictor.py”, line 110, in predict self.forward() File “cmr/nnutils/predictor.py”, line 149, in forward self.cam_pred) File “cmr/nnutils/nmr.py”, line 183, in forward return Render(self.renderer)(verts, faces) File “cmr/nnutils/nmr.py”, line 114, in forward masks = self.renderer.forward_mask(vs, fs) File “cmr/nnutils/nmr.py”, line 50, in forward_mask self.masks = self.renderer.render_silhouettes(self.vertices, self.faces) File “build/bdist.linux-x86_64/egg/neural_renderer/renderer.py”, line 38, in render_silhouettes File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/chainer/functions/array/concat.py”, line 90, in concat y, = Concat(axis).apply(xs) File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/chainer/function_node.py”, line 245, in apply outputs = self.forward(in_data) File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/chainer/functions/array/concat.py”, line 44, in forward return xp.concatenate(xs, self.axis), File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/cupy/manipulation/join.py”, line 49, in concatenate return core.concatenate_method(tup, axis) File “cupy/core/core.pyx”, line 2439, in cupy.core.core.concatenate_method File “cupy/core/core.pyx”, line 2482, in cupy.core.core.concatenate_method File “cupy/core/core.pyx”, line 2533, in cupy.core.core.concatenate File “cupy/core/core.pyx”, line 1630, in cupy.core.core.ndarray.setitem File “cupy/core/core.pyx”, line 3101, in cupy.core.core._scatter_op File “cupy/core/elementwise.pxi”, line 823, in cupy.core.core.ufunc.call File “cupy/util.pyx”, line 39, in cupy.util.memoize.decorator.ret File “cupy/core/elementwise.pxi”, line 622, in cupy.core.core._get_ufunc_kernel File “cupy/core/elementwise.pxi”, line 33, in cupy.core.core._get_simple_elementwise_kernel File “cupy/core/carray.pxi”, line 170, in cupy.core.core.compile_with_cache File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/cupy/cuda/compiler.py”, line 123, in compile_with_cache base = _preprocess(‘’, options, arch) File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/cupy/cuda/compiler.py”, line 86, in _preprocess result = prog.compile(options) File “/home/ubuntu/akshay/cmr/venv_cmr/local/lib/python2.7/site-packages/cupy/cuda/compiler.py”, line 233, in compile raise CompileException(log, self.src, self.name, options) cupy.cuda.compiler.CompileException: nvrtc: error: failed to load builtins

I tried the solution mentioned in this: https://groups.google.com/forum/#!topic/chainer-jp/GKNe5KY_fm0 yet it is not working

[nikhilaravi]

@charliememory did you use chainer 3.3.0 or the latest version? With chainer==3.3.0 I keep getting an error: RuntimeError: CUDA environment is not correctly set up (see https://github.com/chainer/chainer#installation).cannot import name sqrt_fixed I checked chainer.backends.cuda.available and it returned False. With the new version of chainer, I get the nvrtc error above even though I have set the CUDA_PATH and LD_LIBRARY_PATH as instructed in the cupy documentation.

[ShougangShen]

python2.7, CUDA8.0, chainer 3.3.0, cupy 2.3.0, pytorch 0.3.1 works for me. Before running the demo command, I run: export PATH="$PATH:/usr/local/cuda-8.0/bin" ， export LD_LIBRARY_PATH="/usr/local/cuda-8.0/lib64" and reinstall the chainer and cupy.