Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

jaxlib build w/ cuda: File not found during compilation

See original GitHub issue

I’m compiling jaxlib with CUDA 10.0 on Ubuntu 18.04. The build fails with the following error:

$ python3 build/build.py --enable_cuda --cuda_path /usr/local/cuda-10.0/ --cudnn_path /usr/local/cuda-10.0/ --enable_march_native

[...]

ERROR: /home/clem/.cache/bazel/_bazel_clem/ffaac3f7c6ad1cb26f04f1933452eef6/external/nccl_archive/BUILD.bazel:53:1: error while parsing .d file: /h
ome/clem/.cache/bazel/_bazel_clem/ffaac3f7c6ad1cb26f04f1933452eef6/execroot/__main__/bazel-out/k8-opt/bin/external/nccl_archive/_objs/device_lib/pr
od_i32_reduce_scatter.cu.d (No such file or directory)
nvcc fatal   : Could not open input file /tmp/tmpxft_00000004_00000000-6_prod_i32_reduce_scatter.cu.compute_35.cpp1.ii
Target //build:install_xla_in_source_tree failed to build
INFO: Elapsed time: 278.116s, Critical Path: 69.60s
INFO: 1281 processes: 1281 linux-sandbox.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
Traceback (most recent call last):
  File "build/build.py", line 331, in <module>
    main()
  File "build/build.py", line 326, in main
    [":install_xla_in_source_tree", os.getcwd()])
  File "build/build.py", line 50, in shell
    output = subprocess.check_output(cmd)
  File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['./bazel-0.24.1-linux-x86_64', 'run', '--verbose_failures=true', '--config=opt', '--config=mkl_open_source
_only', '--config=cuda', ':install_xla_in_source_tree', '/home/clem/git/jax/build']' returned non-zero exit status 1.

Above this error message are only compiler warnings but no errors which could lead to some file not being created. Am I missing something? Or might there be a file name bug? Thanks a lot for your help!

I’m on a fresh Ubuntu 18.04.2 install with CUDA 10.0, cudnn and driver version 410.48. Full log

Issue Analytics

State:
Created 4 years ago
Comments:7 (7 by maintainers)

Top GitHub Comments

1reaction

hawkinspcommented, Aug 2, 2019

PR #1096 seems to have fixed this for me. Hope that helps!

1reaction

clemischcommented, Jul 8, 2019

Thank you @kroq-gar78 for the information. I posted a separate issue on that as it apparently does not correlate to the compilation error.

Edit: It indeed does not, see #993.

Top Results From Across the Web

Building from source - JAX documentation

To build jaxlib with CUDA support, use python build/build.py --enable_cuda ... repository may include fixes not present in the upstream repository.

Set up environment for JAX sampling with GPU support in ...

Hi everyone, This week, I have spent sometimes to re-install my dev environment, as I need to change to a new hard-drive.

JAX installation with Nvidia CUDA and cudNN support (Fixing ...

This video is all you need to install JAX with Nvidia CUDA and cudNN support in your Python 3.x installation. This video covers...

trax-ml/community - Gitter

After uninstalling jax and jaxlib and installing with the following ... on a linux docker container (Cuda compilation tools, release 11.5, V11.5.119, Build...

build - Compiling jaxlib on Windows 11 - Stack Overflow

7 SET CUDNN_INSTALL_PATH=C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7 SET INCLUDE=C:\Program Files (x86)\Microsoft Visual Studio\ ...