import cupy eats ~1.8Gb of virtual memory for nothing in every process instance

My app (https://github.com/iperov/DeepFaceLive) uses multiple python subprocesses (multiprocessing.Process) to do the work. Every process which imports cupy uses 1.8Gb+ physical memory or paging file.

import cupy

https://user-images.githubusercontent.com/8076202/126863232-90bd879a-dedc-48a6-94c9-bfc8a3e3ee7f.mp4

If user has no free space in paging file, MemoryError will occur.

Can you fix it?

Temporary solution is to import cupy on demand, but the code looks like crap, and unable to use type hinting like img : cp.ndarray, because it requires cupy to be imported.

>>> cupy.show_config()
OS                           : Windows-10-10.0.19041-SP0
CuPy Version                 : 9.0.0
NumPy Version                : 1.21.1
SciPy Version                : 1.5.4
Cython Build Version         : 0.29.23
Cython Runtime Version       : None
CUDA Root                    : D:\DevelopPPP\projects\DeepFaceLive\_internal\CUDA
CUDA Build Version           : 11010
CUDA Driver Version          : 11030
CUDA Runtime Version         : 11010
cuBLAS Version               : 11201
cuFFT Version                : 10300
cuRAND Version               : 10202
cuSOLVER Version             : (11, 0, 0)
cuSPARSE Version             : 11200
NVRTC Version                : (11, 1)
Thrust Version               : 100910
CUB Build Version            : 100910
Jitify Build Version         : 60e9e72
cuDNN Build Version          : 8101
cuDNN Version                : 8005
NCCL Build Version           : None
NCCL Runtime Version         : None
cuTENSOR Version             : None
cuSPARSELt Build Version     : None
Device 0 Name                : NVIDIA GeForce RTX 3090
Device 0 Compute Capability  : 86
Device 0 PCI Bus ID          : 0000:01:00.0

[kmaehashi]

@iperov Please be considerate and respectful to the community and toward other contributors. I understand your frustration but the problem is not simply a CuPy bug but involves how CUDA and Windows work. Discussion is essential to think about how we can deal with this kind of issue.

As I mentioned above, most of the memory usage is coming from CUDA and not CuPy itself. For example, simply loading cuSOLVER (one of CUDA Toolkit library which CuPy depends on) to the process:

>>> import ctypes
>>> ctypes.CDLL(r"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\cusolver64_11.dll")
<CDLL 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\cusolver64_11.dll', handle 7ffc64e00000 at 0x263945c22b0>

takes 650 MB commit charge in Windows. This problem exists in many libraries using CUDA. For example import torch (PyTorch 1.9.0) took 4 GB commit charge in my environment.

I think the workaround available today is to increase the page file size. Although the commit charge is not the amount of memory actually used, Windows reserves the amount from physical memory size + page file size.

We may be able to consider deferring the import of CUDA libraries, but if you call CuPy features in child processes you will experience MemoryError anyway.

[leofang]

Might be related: https://github.com/numpy/numpy/issues/19556.

P.S. please don’t answer in topic if you are not a developer of CuPy.

@iperov What’s the matter with this and the downvotes? Please be mindful and respectful to all constructive discussions, which are all welcomed whether from a CuPy developer or not. @kmaehashi is the CuPy project lead, but it shouldn’t be of concern to you.