Fairseq reinstalls pytorch
See original GitHub issueI have a source-compiled version of pytorch (1.12.0a0+git67ece03
).
When I tried to install fairseq, it uninstall the installed pytorch and install another version (1.12.1+cu102
).
Does anyone know how to resolve the issue?
Here is the log of pip install -e .
of Fairseq:
Obtaining file:///data/home/nyn/packages/fairseq
Installing build dependencies ... done
Checking if build backend supports build_editable ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: tqdm in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (4.64.0)
Requirement already satisfied: cffi in /data/home/nyn/miniconda3/envs/myenv/python3.8/site-packages (from fairseq==0.12.2) (1.15.1)
Requirement already satisfied: omegaconf<2.1 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (2.0.6)
Requirement already satisfied: sacrebleu>=1.4.12 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (2.2.0)
Requirement already satisfied: torchaudio>=0.8.0 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (0.12.1)
Requirement already satisfied: bitarray in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (2.6.0)
Requirement already satisfied: hydra-core<1.1,>=1.0.7 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (1.0.7)
Requirement already satisfied: regex in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (2022.7.25)
Requirement already satisfied: cython in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (0.29.32)
Requirement already satisfied: numpy in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (1.23.1)
Requirement already satisfied: torch in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (1.12.0a0+git67ece03)
Requirement already satisfied: antlr4-python3-runtime==4.8 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from hydra-core<1.1,>=1.0.7->fairseq==0.12.2) (4.8)
Requirement already satisfied: importlib-resources in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from hydra-core<1.1,>=1.0.7->fairseq==0.12.2) (5.9.0)
Requirement already satisfied: typing-extensions in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from omegaconf<2.1->fairseq==0.12.2) (4.3.0)
Requirement already satisfied: PyYAML>=5.1.* in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from omegaconf<2.1->fairseq==0.12.2) (6.0)
Requirement already satisfied: colorama in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from sacrebleu>=1.4.12->fairseq==0.12.2) (0.4.5)
Requirement already satisfied: tabulate>=0.8.9 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from sacrebleu>=1.4.12->fairseq==0.12.2) (0.8.10)
Requirement already satisfied: portalocker in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from sacrebleu>=1.4.12->fairseq==0.12.2) (2.5.1)
Requirement already satisfied: lxml in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from sacrebleu>=1.4.12->fairseq==0.12.2) (4.9.1
)
Collecting torch
Using cached torch-1.12.1-cp38-cp38-manylinux1_x86_64.whl (776.3 MB)
Requirement already satisfied: pycparser in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from cffi->fairseq==0.12.2) (2.21)
Requirement already satisfied: zipp>=3.1.0 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from importlib-resources->hydra-core<1.1,>=1.0.7->fairseq==0.12.2) (3.8.1)
Installing collected packages: torch, fairseq
Attempting uninstall: torch
Found existing installation: torch 1.12.0a0+git67ece03
Uninstalling torch-1.12.0a0+git67ece03:
Successfully uninstalled torch-1.12.0a0+git67ece03
Running setup.py develop for fairseq
Successfully installed fairseq-0.12.2 torch-1.12.1
Issue Analytics
- State:
- Created a year ago
- Reactions:1
- Comments:5 (2 by maintainers)
Top Results From Across the Web
building from source failed · Issue #1804 - GitHub
This seems like an issue with your numpy installation, not fairseq, since it happens when import numpy is called. Please reinstall numpy/pytorch ......
Read more >Failing to install fairseq packages - Google Groups
I am a new user in fairseq . I want to implement this model given in this link https://github.com/LiyuanLucasLiu/Transformer-Clinic.
Read more >Fairseq Transform model not working (Float can't be cast to long)
I've installed python 3.8, pytorch 1.7, and fairseq 0.10.1, on a new machine, then copied in a script and model from a machine...
Read more >Crash when initializing distributed training across 2 machines
I'm running into problems with training (fairseq code) across 2 machines. ... opts) RuntimeError: NCCL error in: /pytorch/torch/lib/c10d/.
Read more >Sockeye 3: Fast Neural Machine Translation with PyTorch
We conduct a reproducible benchmark of PyTorch-based neural machine translation toolkits that includes Sockeye, Fairseq 11 ...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
No, it’s not “regular pip behavior”. The StackOverflow question you pointed us to is for upgrading packages which may require an upgrade of its dependencies. We’re installing an editable version which should obey our currently installed dependencies. This is especially problematic since the version of pytorch isn’t pinned (and therefore installing fairseq shouldn’t do something dumb like installing pytorch built against a different version of CUDA…).
In short, it’s definitively a bug. From the pypa documentation here (emphasis mine):
I think in another issue #4733 , this problem is fully discussed. For my case, adding
--no-build-isolation
will be ok.