[build fail] python: can't open '....': [Errno2] No such file or directory

Please provide the following information. The more we know about your system and use case, the more easily and likely we can help.

Description of the problem / feature request / question:

Trying to build tensorflow with opencl support using this repo. The bazel build is giving me the following error python: can't open file 'external/local_config_sycl/crosstool/computecpp': [Errno 2] No such file or directory

Here’s a verbose output:

ERROR: /home/spike/.cache/bazel/_bazel_spike/ed14cddfddd6d6c9f1aad38aca9f1862/external/farmhash_archive/BUILD.bazel:19:1: C++ compilation of rule '@farmhash_archive//:farmhash' failed (Exit 2): computecpp failed: error executing command 
  (cd /home/spike/.cache/bazel/_bazel_spike/ed14cddfddd6d6c9f1aad38aca9f1862/execroot/org_tensorflow && \
  exec env - \
    COMPUTECPP_TOOLKIT_PATH=/opt/ComputeCpp-CE-0.3.1-Linux \
    HOST_CXX_COMPILER=/usr/bin/g++ \
    HOST_C_COMPILER=/usr/bin/gcc \
    PATH=/home/spike/.nvm/versions/node/v6.10.1/bin:/home/spike/.local/bin:/home/spike/bin:/home/spike/.pyenv/plugins/pyenv-virtualenv/shims:/home/spike/.pyenv/shims:/home/spike/scripts:/home/spike/projects/go:/home/spike/projects/go/bin:/home/spike/.local/bin:/home/spike/bin:/home/spike/.pyenv/plugins/pyenv-virtualenv/shims:/home/spike/.pyenv/shims:/home/spike/scripts:/home/spike/projects/go:/home/spike/projects/go/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/home/spike/.pyenv/versions/ml/bin/python \
    PYTHON_LIB_PATH=/home/spike/.pyenv/versions/ml/lib/python3.5/site-packages \
    TF_NEED_CUDA=0 \
    TF_NEED_OPENCL=1 \
    TF_VECTORIZE_SYCL=1 \
  external/local_config_sycl/crosstool/computecpp -fPIE -fno-omit-frame-pointer -Wall -msse3 -g0 -O2 -DNDEBUG -ffunction-sections -fdata-sections '-march=native' '-std=c++11' '-march=native' -MD -MF bazel-out/local_linux-py3-opt/bin/external/farmhash_archive/_objs/farmhash/external/farmhash_archive/src/farmhash.pic.d '-frandom-seed=bazel-out/local_linux-py3-opt/bin/external/farmhash_archive/_objs/farmhash/external/farmhash_archive/src/farmhash.pic.o' -fPIC -iquote external/farmhash_archive -iquote bazel-out/local_linux-py3-opt/genfiles/external/farmhash_archive -iquote external/bazel_tools -iquote bazel-out/local_linux-py3-opt/genfiles/external/bazel_tools -isystem external/farmhash_archive/src -isystem bazel-out/local_linux-py3-opt/genfiles/external/farmhash_archive/src -isystem external/bazel_tools/tools/cpp/gcc3 -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fno-canonical-system-headers -c external/farmhash_archive/src/farmhash.cc -o bazel-out/local_linux-py3-opt/bin/external/farmhash_archive/_objs/farmhash/external/farmhash_archive/src/farmhash.pic.o).
python: can't open file 'external/local_config_sycl/crosstool/computecpp': [Errno 2] No such file or directory
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 0.313s, Critical Path: 0.08s

When I check inside /home/spike/.cache/bazel/_bazel_spike/ed14cddfddd6d6c9f1aad38aca9f1862/execroot/org_tensorflow the path external/local_config_sycl/crosstool/computecpp exists and I can run it from the command line and I can even compile the file manually using the command that is run by bazel from the same working directory.

Inside the external path I changed all symlinks to real files but still the same error.

How can I debug the python command that is run by bazel in this case ?

If possible, provide a minimal example to reproduce the problem:

Any bazel build --config=sycl command

Environment info

• Operating System: Arch Linux 64

• Bazel version (output of bazel info release):
  both release 0.5.4- (@non-git) and development version (HEAD)

• If bazel info release returns “development version” or “(@non-git)”, please tell us what source tree you compiled Bazel from; git commit hash is appreciated (git rev-parse HEAD):

Have you found anything relevant by searching the web? No

(e.g. StackOverflow answers, GitHub issues, email threads on the bazel-discuss Google group)

Anything else, information or logs or outputs that would be helpful?

(If they are large, please upload as attachment or provide link).

[Taltzipi]

I Ran into a similar issue compiling tensorflow using a pyenv python version, where I received

python: can’t open file ‘external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc’: [Errno 2] No such file or directory

It seems there’s a clash in the use of pyenv and bazel.

What worked for me was:

1. temporarily disable pyenv (commenting out its initialization in .bashrc and opening a new bash) so that ‘which python’ points to the system’s /usr/bin/python,
2. run tensorflow’s configure.py. When asked for the python version, point it to a specific pyenv python version you’ve installed, eg ~/.pyenv/versions/3.6.1/bin/python
3. continue with the bazel build - it should now run successfully.
4. reenable pyenv in bashrc

There’s probably a more elegant way, but worth giving this a try.

[vishnubob]

I am building tensorflow from scratch for the first time, and I immediately hit this issue. I did not want to restart the whole process over again without my carefully constructed pyenv environment, so I looked for a solution. Since the file external/local_config_sycl/crosstool/computecpp is located within our build cache, my (gross) fix was to add a new line to the top of this python script that points directly to the pyenv python executable I wish to run.

(echo "#\!$(pyenv which python)" && cat /PATH/TO/crosstool_wrapper_driver_is_not_gcc) > /tmp/cross
mv /tmp/cross /PATH/TO/crosstool_wrapper_driver_is_not_gcc

I believe the “problem” is the builders’ usage of env - which dumps the environment before it hands the subshell over to external/local_config_sycl/crosstool/computecpp via exec. Since we are in a naked environment, I would assume the pyenv shims aren’t properly initialized, and the shell reports that it can’t find python.