Failed to install ray above 0.6.4 from source in Arm64 architecture

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): ubuntu 18.04 (Arm64 architecture)
• Ray installed from (source or binary): source
• Ray version: 0.6.4
• Python version: 3.6.7
• Exact command to reproduce: pip3 install -e . --verbose

Hi!

I’m trying to install ray 0.6.4 on the NVIDIA Xavier board which has Arm64 CPU.

However, install command fails with following error

Could not find a version that satisfies the requirement pyarrow==0.12.0.RAY (from versions: 0.9.0, 0.10.0, 0.11.0, 0.11.1, 0.12.0, 0.12.1, 0.13.0)
    No matching distribution found for pyarrow==0.12.0.RAY

I found out that build.sh is changed since 0.6.4 so it uses snapshot of pyarrow compiled in X86.

Therefore manually downloading and install whl didn’t work.

Is there a workaround for this problem?

If pyarrow==0.12.0.RAY is same as pyarrow 0.12, can I install it and skip this part in build.sh?

[sammuli]

Well, that was quite a rabbit hole I had to follow, but I finally got it built on my power 9 system.

Some notes on the process:

I followed the instructions given by @pcmoritz above:

git clone https://github.com/apache/arrow/
cd arrow/cpp
mkdir build
cd build
cmake -DARROW_PLASMA=on -DARROW_PYTHON=on -DARROW_DEPENDENCY_SOURCE=BUNDLED ..
make -j8
sudo make install

Then (note the additional ARROW_HOME variable I needed to set, though there’s probably a more elegant way of setting that with cmake):

cd arrow/python
PYARROW_WITH_PLASMA=1  ARROW_HOME=/usr/local python setup.py bdist_wheel

I had an older version of bazel, but that was no good, so I followed the instructions here (https://docs.bazel.build/versions/master/install-compile-source.html) and bootstrapped from a zip distribution of version 0.21.0.

Then finally, I had an outdated version of git, 1.8, (since I’m running on centos 7.4) which didn’t have the command line flags that the ray build was expecting. So, I built a bleeding edge version (2.9.5).

I was then able to do

pip install -e . --verbose

The only other gotcha I encountered was needing to put the arrow .so files into my LD_LIBRARY_PATH, though I suspect that I didn’t quite build arrow right. There’s probably some rpath flag or another that might help that issue.

From there I was able to do a minimal “ray.put” and “ray.get” test and it seemed to work.

[stale[bot]]

Hi again! The issue will be closed because there has been no more activity in the 14 days since the last message.

Please feel free to reopen or open a new issue if you’d still like it to be addressed.

Again, you can always ask for help on our discussion forum or Ray’s public slack channel.

Thanks again for opening the issue!