question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Fairseq reinstalls pytorch

See original GitHub issue

I have a source-compiled version of pytorch (1.12.0a0+git67ece03). When I tried to install fairseq, it uninstall the installed pytorch and install another version (1.12.1+cu102). Does anyone know how to resolve the issue? Here is the log of pip install -e . of Fairseq:

Obtaining file:///data/home/nyn/packages/fairseq
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: tqdm in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (4.64.0)
Requirement already satisfied: cffi in /data/home/nyn/miniconda3/envs/myenv/python3.8/site-packages (from fairseq==0.12.2) (1.15.1)
Requirement already satisfied: omegaconf<2.1 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (2.0.6)
Requirement already satisfied: sacrebleu>=1.4.12 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (2.2.0)
Requirement already satisfied: torchaudio>=0.8.0 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (0.12.1)
Requirement already satisfied: bitarray in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (2.6.0)
Requirement already satisfied: hydra-core<1.1,>=1.0.7 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (1.0.7)
Requirement already satisfied: regex in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (2022.7.25)
Requirement already satisfied: cython in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (0.29.32)
Requirement already satisfied: numpy in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (1.23.1)
Requirement already satisfied: torch in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from fairseq==0.12.2) (1.12.0a0+git67ece03)
Requirement already satisfied: antlr4-python3-runtime==4.8 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from hydra-core<1.1,>=1.0.7->fairseq==0.12.2) (4.8)
Requirement already satisfied: importlib-resources in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from hydra-core<1.1,>=1.0.7->fairseq==0.12.2) (5.9.0)
Requirement already satisfied: typing-extensions in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from omegaconf<2.1->fairseq==0.12.2) (4.3.0)
Requirement already satisfied: PyYAML>=5.1.* in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from omegaconf<2.1->fairseq==0.12.2) (6.0)
Requirement already satisfied: colorama in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from sacrebleu>=1.4.12->fairseq==0.12.2) (0.4.5)
Requirement already satisfied: tabulate>=0.8.9 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from sacrebleu>=1.4.12->fairseq==0.12.2) (0.8.10)
Requirement already satisfied: portalocker in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from sacrebleu>=1.4.12->fairseq==0.12.2) (2.5.1)
Requirement already satisfied: lxml in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from sacrebleu>=1.4.12->fairseq==0.12.2) (4.9.1
)
Collecting torch
  Using cached torch-1.12.1-cp38-cp38-manylinux1_x86_64.whl (776.3 MB)
Requirement already satisfied: pycparser in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from cffi->fairseq==0.12.2) (2.21)
Requirement already satisfied: zipp>=3.1.0 in /data/home/nyn/miniconda3/envs/myenv/lib/python3.8/site-packages (from importlib-resources->hydra-core<1.1,>=1.0.7->fairseq==0.12.2) (3.8.1)
Installing collected packages: torch, fairseq
  Attempting uninstall: torch
    Found existing installation: torch 1.12.0a0+git67ece03
    Uninstalling torch-1.12.0a0+git67ece03:
      Successfully uninstalled torch-1.12.0a0+git67ece03
  Running setup.py develop for fairseq
Successfully installed fairseq-0.12.2 torch-1.12.1

Issue Analytics

  • State:open
  • Created a year ago
  • Reactions:1
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
eripcommented, Aug 7, 2022

No, it’s not “regular pip behavior”. The StackOverflow question you pointed us to is for upgrading packages which may require an upgrade of its dependencies. We’re installing an editable version which should obey our currently installed dependencies. This is especially problematic since the version of pytorch isn’t pinned (and therefore installing fairseq shouldn’t do something dumb like installing pytorch built against a different version of CUDA…).

In short, it’s definitively a bug. From the pypa documentation here (emphasis mine):

When your project is installed (e.g., using pip), all of the dependencies not already installed will be located (via PyPI), downloaded, built (if necessary), and installed

0reactions
weiyx16commented, Oct 22, 2022

I think in another issue #4733 , this problem is fully discussed. For my case, adding --no-build-isolation will be ok.

Read more comments on GitHub >

github_iconTop Results From Across the Web

building from source failed · Issue #1804 - GitHub
This seems like an issue with your numpy installation, not fairseq, since it happens when import numpy is called. Please reinstall numpy/pytorch ......
Read more >
Failing to install fairseq packages - Google Groups
I am a new user in fairseq . I want to implement this model given in this link https://github.com/LiyuanLucasLiu/Transformer-Clinic.
Read more >
Fairseq Transform model not working (Float can't be cast to long)
I've installed python 3.8, pytorch 1.7, and fairseq 0.10.1, on a new machine, then copied in a script and model from a machine...
Read more >
Crash when initializing distributed training across 2 machines
I'm running into problems with training (fairseq code) across 2 machines. ... opts) RuntimeError: NCCL error in: /pytorch/torch/lib/c10d/.
Read more >
Sockeye 3: Fast Neural Machine Translation with PyTorch
We conduct a reproducible benchmark of PyTorch-based neural machine translation toolkits that includes Sockeye, Fairseq 11 ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found