python error with faiss on GPU with cuda despite successful installation

Question

I'm working on a Google Cloud VM with CUDA 12. I tried to install either faiss-gpu-cu12 or faiss-gpu-cu12[fix_cuda] using either a venv or pyenv virtual environment, under python 3.12.4. For the application we also need langchain packages as below:

python --version
Python 3.12.4

python -m pip install langchain langchain-huggingface langchain-community faiss-gpu-cu12[fix_cuda]

pip gives no error and everything looks fine, but when I run the minimal example below:

from langchain_community.vectorstores import FAISS
from langchain_huggingface import HuggingFaceEmbeddings
from sentence_transformers import SentenceTransformer

docs = [ 'sefsl;fk lskdf;lk s', 'ewrl kwelklekfl ls ;' ]
embedder = HuggingFaceEmbeddings(model_name='sentence-transformers/all-MiniLM-L6-v2')

f = FAISS.from_texts(docs, embedding=embedder)
embeddings = embedder.encode(docs)

I get this error:

Traceback (most recent call last):
  File "/home/jupyterlab_user/erwan/RAG/.venv/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py", line 55, in dependable_faiss_import
    import faiss
  File "/home/jupyterlab_user/erwan/RAG/.venv/lib/python3.12/site-packages/faiss/__init__.py", line 16, in <module>
    from .loader import *
  File "/home/jupyterlab_user/erwan/RAG/.venv/lib/python3.12/site-packages/faiss/loader.py", line 111, in <module>
    from .swigfaiss import *
  File "/home/jupyterlab_user/erwan/RAG/.venv/lib/python3.12/site-packages/faiss/swigfaiss.py", line 10, in <module>
    from . import _swigfaiss
ImportError: /home/jupyterlab_user/erwan/RAG/.venv/lib/python3.12/site-packages/faiss/_swigfaiss.cpython-312-x86_64-linux-gnu.so: ELF load command address/offset not properly aligned

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jupyterlab_user/erwan/RAG/minimal-faiss-expl2.py", line 9, in <module>
    f = FAISS.from_texts(docs, embedding=embedder)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jupyterlab_user/erwan/RAG/.venv/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py", line 1042, in from_texts
    return cls.__from(
           ^^^^^^^^^^^
  File "/home/jupyterlab_user/erwan/RAG/.venv/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py", line 994, in __from
    faiss = dependable_faiss_import()
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jupyterlab_user/erwan/RAG/.venv/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py", line 57, in dependable_faiss_import
    raise ImportError(
ImportError: Could not import faiss python package. Please install it with `pip install faiss-gpu` (for CUDA supported GPU) or `pip install faiss-cpu` (depending on Python version).

It looks like the Faiss library is not installed properly, even though pip gave no error when installing package faiss-gpu-cu12. Is there anything I missed? For instance, is there any known incompatibility between the packages above?

For now I'm restricted to use faiss-cpu, so any suggestion welcome!

Additional info:

$ pip freeze | grep faiss
faiss-gpu-cu12==1.8.0.2

$ pip freeze | grep langchain
langchain==0.3.3
langchain-community==0.3.2
langchain-core==0.3.12
langchain-huggingface==0.1.0
langchain-text-splitters==0.3.0

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Wed_Jul_22_19:09:09_PDT_2020
Cuda compilation tools, release 11.0, V11.0.221
Build cuda_11.0_bu.TC445_37.28845127_0

you probably would have seen this, github.com/facebookresearch/faiss/blob/… seems to indicate it only supports CUDA 11.4 and 12.1? I would suggest adding your CUDA version into the question. I don't use any of these libraries so feel free to disregard this — python_user
– python_user, Commented Oct 18, 2024 at 15:22
Can you mention the versions of the libraries that you are using? i.e. langchain, langchain-huggingface, langchain-community, faiss-gpu — Ro.oT
– Ro.oT, Commented Oct 18, 2024 at 15:32
@python_user omg I think you found the issue: it can only be installed with conda... Feel free to write an answer telling me to RTFM! — Erwan
– Erwan, Commented Oct 18, 2024 at 16:14
Just a note, as per your recent edit, you seem to be using Cuda 11.0 not 12.0. But you are specifying faiss-gpu-cu12 which is for cuda 12.x versions. Although they recommend to install using conda channels, I was able to install faiss-gpu-cu12 via pip on my linux machine. — Ro.oT
– Ro.oT, Commented Oct 18, 2024 at 18:15
@Erwan I have done so, I will remove the answer if its no longer true — python_user
– python_user, Commented Oct 19, 2024 at 1:43

python_user · Accepted Answer · 2024-10-19 01:43:12Z

1

+400

As per the installation readme for faiss, it currently seems to only support CUDA versions 11.4 and 12.1. Your post indicates you are on CUDA 11.0.

faiss-gpu, containing both CPU and GPU indices, is available on Linux (x86_64 only) for CUDA 11.4 and 12.1

Further, the readme for faiss-gpu-cu12 mentions

The faiss-gpu-cu12 package (the binaries contained in it) is minor version compatible with CUDA and will work dynamically linked with CUDA 12.X (X>=1).

So it seems you have to be on version 12.1 or above if using version 12.

PS: I added this as an answer in response to this.

answered Oct 19, 2024 at 1:43

python_user

7,2482 gold badges18 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Erwan Over a year ago

Thank you, at least this explains my issue. I tried with conda but it didn't work, probably due to the cuda version, indeed. I will check with colleagues on Monday if this can be fixed.

Erwan Over a year ago

I tried with the right cuda version, in every possible variant of the environment, and even with the help of two colleagues, we were unable to make it work, unfortunately... I give up, but your answer was still the best I received!

python_user Over a year ago

@Erwan thanks for being considerate, appreciate it

oppressionslayer · Accepted Answer · 2024-10-20 06:27:48Z

1

First check this since your running python12:

pyenv install 3.10.11  # Example Python version
pyenv virtualenv 3.10.11 faiss-env
pyenv activate faiss-env

And did you try this and is cuda necessary?

For CUDA 12:

pip uninstall faiss-gpu-cu12
pip install faiss-gpu-cu12[fix_cuda]

answered Oct 20, 2024 at 6:27

oppressionslayer

7,2242 gold badges11 silver badges26 bronze badges

Comments

Abdulkadir Ugas · Accepted Answer · 2025-02-19 06:10:12Z

1

if you are facing an error which is numpy 2.0 crashing error you can try to install using conda as per INSTALL.md instead of pip that is what work for me.

# CPU-only version
$ conda install -c pytorch faiss-cpu

# GPU(+CPU) version
$ conda install -c pytorch -c nvidia faiss-gpu

answered Feb 19 at 6:10

Abdulkadir Ugas

5115 silver badges17 bronze badges

Collectives™ on Stack Overflow

python error with faiss on GPU with cuda despite successful installation

3 Answers 3

3 Comments

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

Comments

Related