faiss-gpu: index_cpu_to_gpu() hangs (doesn't occur with the conda package)
See original GitHub issueWhen I install faiss-gpu via pip, index_cpu_to_gpu() seems to hang forever. For example, this code sample hangs for me:
import faiss
index_flat = faiss.IndexFlatL2(16)
gpu_index_flat = faiss.index_cpu_to_gpu(faiss.StandardGpuResources(), 0, index_flat)
The index_cpu_to_gpu() will hang, spinning a CPU at 100% seemingly forever, mostly in libnvidia-ptxjitcompiler.so.510.54
.
I confirmed this on two different machines with different GPUs (A100 and A10G).
I found two workarounds:
- Downgrade to faiss==1.5.3. I hit the problem with 1.6.0 or later.
- Use the conda package. I don’t observe the issue with conda, even with the latest version of faiss-gpu (1.7.2)
Maybe faiss-gpu wheel isn’t built with CUDA 11 support, and that’s why it doesn’t work with A100 / A10G?
Issue Analytics
- State:
- Created a year ago
- Comments:17 (7 by maintainers)
Top Results From Across the Web
FAISS-GPU cannot be used with GPU version 30xx
I'm writing Python source code and using Faiss. I can use Faiss on CPU, 20xx GPU eg: ...
Read more >faiss-gpu - PyPI
A library for efficient similarity search and clustering of dense vectors.
Read more >Faiss Users | # Faiss 1.6.3 is out - Facebook
Faiss 1.6.3 is out! This release contains mainly extensions to existing indexes and fixes. ## What's new? On GPU: * Support alternative distances......
Read more >Faiss Gpu - :: Anaconda.org
Faiss is a library for efficient similarity search and clustering of dense vectors. It contains algorithms that search in sets of vectors of...
Read more >Introduction to Facebook AI Similarity Search (Faiss) - Pinecone
Now, Faiss not only allows us to build an index and search — but ... So, CUDA-enabled Linux users, type conda install -c...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Hey everyone!
Thanks a lot for this chain! My issue got solved!
Issue: I was not able to use FAISS on the NVIDIA A100. The FAISS index was not getting pushed to the GPU!
For people who want a solution at one place. Please follow these steps below:
As faiss-gpu 1.7.3 is not available on pip. You can download the artifact directory (a directory containing multiple wheel files based on different systems) from here -> https://github.com/kyamagu/faiss-wheels/actions/runs/3487300515 . Go to bottom of the page -> locate artifacts and download it to your machine.
Once you have the artifacts folder, you need to see which file is the most suitable for you system. For example: I used the following file: faiss_gpu-1.7.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl . The filename itself indicates the configurations of a system. In the filename cp38 could mean CPython 3.8 (that is what I think). The artifacts directory also has many wheels for different systems. Locate the one compatible for your machine.
Once you have located your compatible wheel just run the following:
python -m pip install /path/to/wheel/faiss_gpu-1.7.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
And you are good to go!
Thanks to @kyamagu for maintaining the repo!
OK, yeah, that seems to work! (Not sure how that’s different from what I was trying previously.)
That wheel seems to fix my trivial repro, so it addresses the issue, at least as far as I can tell. Presumably this particular wheel won’t work for someone who has CUDA 10, but that’s a different problem. That’s why PyTorch versions are so complicated.