RTX 3080 compatibility OSError: libcudart.so.10.2: cannot open shared object file: No such file or directory

📚 Installation

Environment

• OS: Linux 5.8.0-50-generic #56~20.04.1-Ubuntu
• Python version: 3.8.10
• PyTorch version: 1.8.1
• CUDA/cuDNN version: 11.1
• GCC version: 9.3.0
• How did you try to install PyTorch Geometric and its extensions (wheel, source): wheel
• Any other relevant information: GeForce RTX 3080 with CUDA capability sm_86

Checklist

• I followed the installation guide.
• I cannot find my error message in the FAQ.
• I set up CUDA correctly and can compile CUDA code via nvcc.
• I do have multiple CUDA versions on my machine.

Additional context

nvidia-smi shows cuda version 11.2 and I tried 10.2 here and got this message when trying to use GPU.

torch/cuda/__init__.py:104: UserWarning: 
GeForce RTX 3080 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75 compute_37.
If you want to use the GeForce RTX 3080 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/
  warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))

I then tried 11.1 but now on import I’m getting the below

Torch version: 1.8.1
Cuda available: True
Cuda version: 11.1
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-9-f74940949094> in <module>
      3 print(f"Cuda version: {torch.version.cuda}")
      4 
----> 5 import torch_geometric
      6 print(f"Torch geometric version: {torch_geometric.__version__}")
/vol/biomedic3/nsm116/pygeo3/lib/python3.8/site-packages/torch_geometric/__init__.py in <module>
      3 
      4 from .debug import is_debug_enabled, debug, set_debug
----> 5 import torch_geometric.data
      6 import torch_geometric.transforms
      7 import torch_geometric.utils
/vol/biomedic3/nsm116/pygeo3/lib/python3.8/site-packages/torch_geometric/data/__init__.py in <module>
----> 1 from .data import Data
      2 from .temporal import TemporalData
      3 from .batch import Batch
      4 from .dataset import Dataset
      5 from .in_memory_dataset import InMemoryDataset
/vol/biomedic3/nsm116/pygeo3/lib/python3.8/site-packages/torch_geometric/data/data.py in <module>
      6 import torch
      7 import torch_geometric
----> 8 from torch_sparse import coalesce, SparseTensor
      9 from torch_geometric.utils import (contains_isolated_nodes,
     10                                    contains_self_loops, is_undirected)
/vol/biomedic3/nsm116/pygeo3/lib/python3.8/site-packages/torch_sparse/__init__.py in <module>
     12         '_saint', '_sample', '_relabel'
     13 ]:
---> 14     torch.ops.load_library(importlib.machinery.PathFinder().find_spec(
     15         f'{library}_{suffix}', [osp.dirname(__file__)]).origin)
     16 
/vol/biomedic3/nsm116/pygeo3/lib/python3.8/site-packages/torch/_ops.py in load_library(self, path)
    102             # static (global) initialization code in order to register custom
    103             # operators with the JIT.
--> 104             ctypes.CDLL(path)
    105         self.loaded_libraries.add(path)
    106 
/vol/biomedic3/nsm116/pygeo3/lib/python3.8/ctypes/__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error, winmode)
    379 
    380         if handle is None:
--> 381             self._handle = _dlopen(self._name, mode)
    382         else:
    383             self._handle = handle
OSError: libcudart.so.10.2: cannot open shared object file: No such file or directory

even after

export PATH=/vol/cuda/11.1.0-cudnn8.0.4.30/bin:$PATH
export CPATH=/vol/cuda/11.1.0-cudnn8.0.4.30/include:$CPATH
export LD_LIBRARY_PATH=/vol/cuda/11.1.0-cudnn8.0.4.30/lib64:$LD_LIBRARY_PATH

No wheels for nightly PyTorch

Any suggestions please?

[scarpma]

Oh, sorry, I didn’t know what a wheel is. By the way I use rtx 3090 and it works fine. Sorry I can’t help…

[nairouzshehata]

Somehow, PyTorch is still looking for CUDA 10.2 when loading the extension package. This might be fixable by also setting $CUDA_HOME to your CUDA 11.1 path.

I created a fresh environment (installed packages as described to @scarpma) and instead of setting the paths I run the cuda setup.sh and all is good now

Torch version: 1.8.0 Cuda version: 11.1 Torch geometric version: 1.7.0 Python: 3.8.10

Thanks @rusty1s and @scarpma !