win10，cupy and pytorch can‘t work together

torch can work on project cupy112 also can work without torch

C:>python
Python 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import cupy
>>> cupy.show_config()
OS                           : Windows-10-10.0.19043-SP0
Python Version               : 3.8.10
CuPy Version                 : 9.4.0
CuPy Platform                : NVIDIA CUDA
NumPy Version                : 1.19.3
SciPy Version                : 1.7.1
Cython Build Version         : 0.29.24
Cython Runtime Version       : None
CUDA Root                    : C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2
nvcc PATH                    : C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin\nvcc.EXE
CUDA Build Version           : 11020
CUDA Driver Version          : 11040
CUDA Runtime Version         : 11020
cuBLAS Version               : (available)
cuFFT Version                : 10401
cuRAND Version               : 10203
cuSOLVER Version             : (11, 1, 0)
cuSPARSE Version             : (available)
NVRTC Version                : (11, 2)
Thrust Version               : 101000
CUB Build Version            : 101000
Jitify Build Version         : 60e9e72
cuDNN Build Version          : 8202
cuDNN Version                : 8201
NCCL Build Version           : None
NCCL Runtime Version         : None
cuTENSOR Version             : 10301
cuSPARSELt Build Version     : None
Device 0 Name                : NVIDIA GeForce RTX 3070
Device 0 Compute Capability  : 86
Device 0 PCI Bus ID          : 0000:05:00.0
>>>

i tried install cupy like pip install cupy-cuda112 and pip install cupy-cuda112 -f https://github.com/cupy/cupy/releases/tag/v10.0.0b2

C:\>pip install cupy-cuda112 -f https://github.com/cupy/cupy/releases/tag/v10.0.0b2
WARNING: Ignoring invalid distribution -upy-cuda112 (c:\users\optoma\appdata\roaming\python\python38\site-packages)
WARNING: Ignoring invalid distribution - (c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages)
WARNING: Ignoring invalid distribution -upy-cuda112 (c:\users\optoma\appdata\roaming\python\python38\site-packages)
WARNING: Ignoring invalid distribution - (c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages)
Looking in links: https://github.com/cupy/cupy/releases/tag/v10.0.0b2
Collecting cupy-cuda112
  Using cached cupy_cuda112-9.4.0-cp38-cp38-win_amd64.whl (59.9 MB)
Requirement already satisfied: fastrlock>=0.5 in c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages (from cupy-cuda112) (0.6)
Requirement already satisfied: numpy<1.24,>=1.17 in c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages (from cupy-cuda112) (1.19.3)
WARNING: Ignoring invalid distribution -upy-cuda112 (c:\users\optoma\appdata\roaming\python\python38\site-packages)
WARNING: Ignoring invalid distribution - (c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages)
Installing collected packages: cupy-cuda112
WARNING: Ignoring invalid distribution -upy-cuda112 (c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages)
WARNING: Ignoring invalid distribution - (c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages)
Successfully installed cupy-cuda112-9.4.0
WARNING: Ignoring invalid distribution -upy-cuda112 (c:\users\optoma\appdata\roaming\python\python38\site-packages)
WARNING: Ignoring invalid distribution - (c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages)
WARNING: Ignoring invalid distribution -upy-cuda112 (c:\users\optoma\appdata\roaming\python\python38\site-packages)
WARNING: Ignoring invalid distribution - (c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages)
WARNING: Ignoring invalid distribution -upy-cuda112 (c:\users\optoma\appdata\roaming\python\python38\site-packages)
WARNING: Ignoring invalid distribution - (c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages)
WARNING: You are using pip version 21.1.1; however, version 21.2.4 is available.
You should consider upgrading via the 'c:\users\optoma\appdata\local\programs\python\python38\python.exe -m pip install --upgrade pip' command.

but when uninstall it，got some err

C:>pip uninstall cupy-cuda112
WARNING: Ignoring invalid distribution -upy-cuda112 (c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages)
WARNING: Ignoring invalid distribution -upy-cuda112 (c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages)
Found existing installation: cupy-cuda112 9.4.0
Uninstalling cupy-cuda112-9.4.0:
  Would remove:
    c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\cupy\*
    c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\cupy_backends\*
    c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\cupy_cuda112-9.4.0.dist-info\*
    c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\cupyx\*
Proceed (y/n)? y
  Successfully uninstalled cupy-cuda112-9.4.0
ERROR: Exception:
Traceback (most recent call last):
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\pip\_internal\cli\base_command.py", line 180, in _main
    status = self.run(options, args)
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\pip\_internal\commands\uninstall.py", line 89, in run
    uninstall_pathset.commit()
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\pip\_internal\req\req_uninstall.py", line 442, in commit
    self._moved_paths.commit()
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\pip\_internal\req\req_uninstall.py", line 282, in commit
    save_dir.cleanup()
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\pip\_internal\utils\temp_dir.py", line 184, in cleanup
    rmtree(self._path)
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\tenacity\__init__.py", line 339, in wrapped_f
    return self(f, *args, **kw)
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\tenacity\__init__.py", line 430, in __call__
    do = self.iter(retry_state=retry_state)
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\tenacity\__init__.py", line 378, in iter
    raise retry_exc.reraise()
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\tenacity\__init__.py", line 206, in reraise
    raise self.last_attempt.result()
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\concurrent\futures\_base.py", line 437, in result
    return self.__get_result()
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\concurrent\futures\_base.py", line 389, in __get_result
    raise self._exception
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\tenacity\__init__.py", line 433, in __call__
    result = fn(*args, **kwargs)
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\site-packages\pip\_internal\utils\misc.py", line 135, in rmtree
    shutil.rmtree(dir, ignore_errors=ignore_errors, onerror=rmtree_errorhandler)
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\shutil.py", line 740, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\shutil.py", line 613, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\shutil.py", line 613, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\shutil.py", line 618, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "c:\users\optoma\appdata\local\programs\python\python38\lib\shutil.py", line 616, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 5] 拒绝访问。: 'C:\\Users\\OPTOMA\\AppData\\Local\\Programs\\Python\\Python38\\Lib\\site-packages\\~-py\\.data\\lib\\nvToolsExt64_1.dll'

nvcc -V

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Sun_Feb_14_22:08:44_Pacific_Standard_Time_2021
Cuda compilation tools, release 11.2, V11.2.152
Build cuda_11.2.r11.2/compiler.29618528_0

i didn’t find torch-cu11.2，so torch-cu11.1 was installed， it works。

C:\Users\OPTOMA>python
Python 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> print(torch.backends.cudnn.version())
8005
>>> print(torch.__version__)
1.9.0+cu111
>>> print(torch.version.cuda)
11.1

import torch after import cupy：

Python 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import cupy
>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\OPTOMA\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\__init__.py", line 124, in <module>
    raise err
OSError: [WinError 127] 找不到指定的程序。 Error loading "C:\Users\OPTOMA\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\lib\cublas64_11.dll" or one of its dependencies.
>>> exit()

cublas64_11.dll can found in the path

import cupy after import torch：

Python 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import cupy
Traceback (most recent call last):
  File "C:\Users\OPTOMA\AppData\Local\Programs\Python\Python38\lib\site-packages\cupy\__init__.py", line 16, in <module>
    from cupy import _core  # NOQA
  File "C:\Users\OPTOMA\AppData\Local\Programs\Python\Python38\lib\site-packages\cupy\_core\__init__.py", line 1, in <module>
    from cupy._core import core  # NOQA
  File "cupy\_core\core.pyx", line 1, in init cupy._core.core
  File "C:\Users\OPTOMA\AppData\Local\Programs\Python\Python38\lib\site-packages\cupy\cuda\__init__.py", line 8, in <module>
    from cupy.cuda import compiler  # NOQA
  File "C:\Users\OPTOMA\AppData\Local\Programs\Python\Python38\lib\site-packages\cupy\cuda\compiler.py", line 11, in <module>
    from cupy.cuda import device
  File "cupy\cuda\device.pyx", line 1, in init cupy.cuda.device
ImportError: DLL load failed while importing runtime: 找不到指定的程序。

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\OPTOMA\AppData\Local\Programs\Python\Python38\lib\site-packages\cupy\__init__.py", line 37, in <module>
    raise ImportError(_msg) from e
ImportError: CuPy is not correctly installed.

If you are using wheel distribution (cupy-cudaXX), make sure that the version of CuPy you installed matches with the version of CUDA on your host.
Also, confirm that only one CuPy package is installed:
  $ pip freeze

If you are building CuPy from source, please check your environment, uninstall CuPy and reinstall it with:
  $ pip install cupy --no-cache-dir -vvvv

Check the Installation Guide for details:
  https://docs.cupy.dev/en/latest/install.html

original error: DLL load failed while importing runtime: 找不到指定的程序。
>>> exit()

Any help is appreciated！！！

[aja9675]

Makes sense! Thanks for the assistance. I will file the bug over on PyTorch.

[aja9675]

I’m runinng Pytorch ‘1.9.1+cu111’. It’s working just fine with the CUDA 11.4 toolkit:

nvcc -V nvcc: NVIDIA ® Cuda compiler driver Copyright © 2005-2021 NVIDIA Corporation Built on Sun_Aug_15_21:14:11_PDT_2021 Cuda compilation tools, release 11.4, V11.4.120 Build cuda_11.4.r11.4/compiler.30300941_0

And as long as I import cupy before torch, I can run inference on my model successfully.

I would not expect the import order to matter. If there’s a reason for this, please let me know.