Segfault when copying a non-contiguous CuPy array to host
See original GitHub issue>>> import cupy as cp
>>>
>>> ptr = cp.cuda.alloc(2**27 * 8)
>>> a = cp.ndarray(shape=(2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2),\
... strides=(128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768, 65536, 131072, 262144, 524288, 1048576, 2097152, 4194304, 8388608, 16777216, 33554432, 67108864, 134217728, 536870912, 268435456, 8, 16, 32, 64),\
... memptr=ptr)
>>>
>>> b = cp.asnumpy(a)
Segmentation fault (core dumped)
It either segfaults hard, or causes CUDA_ERROR_MISALIGNED_ADDRESS
in a real workload:
Traceback (most recent call last):
File "<string>", line 23, in <module>
File "/home/leof/miniforge3/envs/opt_einsum_dev/lib/python3.8/site-packages/cupy/__init__.py", line 773, in asnumpy
return a.get(stream=stream, order=order)
File "cupy/_core/core.pyx", line 1567, in cupy._core.core.ndarray.get
File "cupy/_core/core.pyx", line 1623, in cupy._core.core.ndarray.get
File "cupy/_core/core.pyx", line 1625, in cupy._core.core.ndarray.get
File "cupy/_core/core.pyx", line 2404, in cupy._core.core._internal_ascontiguousarray
File "cupy/_core/_kernel.pyx", line 1185, in cupy._core._kernel.ufunc.__call__
File "cupy/_core/_kernel.pyx", line 1212, in cupy._core._kernel.ufunc._get_ufunc_kernel
File "cupy/_core/_kernel.pyx", line 952, in cupy._core._kernel._get_ufunc_kernel
File "cupy/_core/_kernel.pyx", line 62, in cupy._core._kernel._get_simple_elementwise_kernel
File "cupy/_core/core.pyx", line 2030, in cupy._core.core.compile_with_cache
File "/home/leof/miniforge3/envs/opt_einsum_dev/lib/python3.8/site-packages/cupy/cuda/compiler.py", line 460, in compile_with_cache
return _compile_with_cache_cuda(
File "/home/leof/miniforge3/envs/opt_einsum_dev/lib/python3.8/site-packages/cupy/cuda/compiler.py", line 529, in _compile_with_cache_cuda
mod.load(cubin)
File "cupy/cuda/function.pyx", line 241, in cupy.cuda.function.Module.load
File "cupy/cuda/function.pyx", line 243, in cupy.cuda.function.Module.load
File "cupy_backends/cuda/api/driver.pyx", line 253, in cupy_backends.cuda.api.driver.moduleLoadData
File "cupy_backends/cuda/api/driver.pyx", line 125, in cupy_backends.cuda.api.driver.check_status
cupy_backends.cuda.api.driver.CUDADriverError: CUDA_ERROR_MISALIGNED_ADDRESS: misaligned address
Traceback (most recent call last):
File "cupy_backends/cuda/api/driver.pyx", line 260, in cupy_backends.cuda.api.driver.moduleUnload
File "cupy_backends/cuda/api/driver.pyx", line 125, in cupy_backends.cuda.api.driver.check_status
cupy_backends.cuda.api.driver.CUDADriverError: CUDA_ERROR_MISALIGNED_ADDRESS: misaligned address
Exception ignored in: 'cupy.cuda.function.Module.__dealloc__'
(... repeat the above traceback N times ...)
By the way, technically this array is illegal in CuPy (a.ndim=27
> MAX_NDIM=25
, see #4193), but I am not sure if this is relevant.
Issue Analytics
- State:
- Created 2 years ago
- Reactions:1
- Comments:5 (5 by maintainers)
Top Results From Across the Web
Read/write a CuPy array on multiple GPUs - Stack Overflow
I have a large array. I need to process parts of this array on multiple GPUs to not copy the same array to...
Read more >maximum ndim discrepancy between CuPy and NumPy ...
Question: maximum ndim discrepancy between CuPy and NumPy? #4193 ... Segfault when copying a non-contiguous CuPy array to host #5721.
Read more >Basics of CuPy — CuPy 11.4.0 documentation
array (arr, dtype, copy=False) . Move array from a device to the host#. Moving a device array to the host can be done...
Read more >MPI for Python - Read the Docs
This document describes the MPI for Python package. MPI for Python provides Python bindings for the Message.
Read more >10. Release Notes — Numba 0.47.0-py3.6-macosx ... - PyData |
Version 0.47.0 (Jan 2, 2020)¶. This release expands the capability of Numba in a number of important areas and is also significant as...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Hi @emcastillo, could you reopen and assign this issue to me (as I can’t reopen)? I’d like to add a safe check so that we don’t accidentally compile a copy kernel that can’t do its job correctly.
Since this seems to be a duplicate, let’s close this one. Feel free to reopen if you think it is not 😃