Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

numpy version conflicts

See original GitHub issue

install from setup.py (numpy version 1.21.6)

~/alpa$ python3 tests/test_install.py
RuntimeError: module compiled against API version 0xf but this version of numpy is 0xe

after install numpy=1.22.0

python3 tests/test_install.py
Traceback (most recent call last):
  File "tests/test_install.py", line 13, in <module>
    from alpa import (init, parallelize, grad, ShardParallel, automatic_layer_construction,
  File "/home/ubuntu/alpa/alpa/__init__.py", line 1, in <module>
    from alpa.api import (init, shutdown, parallelize, grad, value_and_grad,
  File "/home/ubuntu/alpa/alpa/api.py", line 15, in <module>
    from alpa.parallel_method import ParallelMethod, ShardParallel
  File "/home/ubuntu/alpa/alpa/parallel_method.py", line 26, in <module>
    from alpa.pipeline_parallel.compile_executable import compile_pipeshard_executable
  File "/home/ubuntu/alpa/alpa/pipeline_parallel/compile_executable.py", line 27, in <module>
    from alpa.pipeline_parallel.stage_construction import (
  File "/home/ubuntu/alpa/alpa/pipeline_parallel/stage_construction.py", line 10, in <module>
    import numba
  File "/home/ubuntu/.local/lib/python3.8/site-packages/numba/__init__.py", line 200, in <module>
    _ensure_critical_deps()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/numba/__init__.py", line 140, in _ensure_critical_deps
    raise ImportError("Numba needs NumPy 1.21 or less")
ImportError: Numba needs NumPy 1.21 or less

I disable numba importerror in the end and passed this test, was wondering what would be the potential problems.

Issue Analytics

State:
Created a year ago
Reactions:1
Comments:6 (4 by maintainers)

Top GitHub Comments

1reaction

marwagecommented, Jun 8, 2022

Hi, I had the same issue and I fixed it by removing the version requirement from numpy in setup.py https://github.com/alpa-projects/alpa/blob/3995d72359f78edaf54de7bc8fe9bdd2c2d8d680/setup.py#L88 My current version is numpy==1.22.4. I hope that helps others too.

0reactions

zhisbugcommented, Jun 10, 2022

@huangjundashuaige Hello, has your issue been addressed?

It is because, during the compilation of jaxlib, a higher version of numpy is used, but when running Alpa, a lower version of numpy is used.

This is possible to happen because when running pip install cupy-cuda11x, pip will resolve the dependency and install a newer version of numpy (>1.22)

But alpa’s setup.py declares numpy<1.22.

We have addressed this issue via PR #492 and PR #499