Complex number calculations differ from numpy

Description

Cupy functions (einsum, matmul, correlate) return different results for complex numbers than their numpy equivalents. The order is often small in comparison to the magnitude of the number (on the order of machine precision), however given that the calculations are run with identical inputs I would expect identical outputs.

To Reproduce

import numpy as np
import cupy as cp
import scipy.signal as sp
import cupyx.scipy.signal as xp

### Test of correlate()
l = 300
n = np.random.rand(l) + 1j*np.random.rand(l)
n = np.asarray(n, dtype=np.complex64)
c = cp.asarray(n, dtype=cp.complex64)
print('-------------')
print('n vs c:', np.allclose(n, c.get()))

ncorr = np.correlate(n, n)  # numpy
ccorr = cp.correlate(c, c)  # cupy
scorr = sp.correlate(n, n, mode='valid')  # scipy
xcorr = xp.correlate(c, c, mode='valid')  # cupyx

print('CPU----------')
print('numpy.correlate():', ncorr)
print('scipy.correlate():', scorr)
print('GPU----------')
print('cupy.correlate():', ccorr)
print('cupyx.correlate():', xcorr)
print('-------------')
print('diff:', ncorr-ccorr.get())

### Test of matmul
n = np.random.rand(3, 3) + 1j*np.random.rand(3, 3)
c = cp.asarray(n)
np_matmul = np.matmul(n, n)
cp_matmul = cp.matmul(c, c)
cp_to_np = cp.asnumpy(cp_matmul)
print("Matmul diff: {}".format(np_matmul - cp_to_np))

### Test of einsum
np_einsum = np.einsum('ij,jk->ik', n, n)
cp_einsum = cp.einsum('ij,jk->ik', c, c)
cp_einsum_cpu = cp.asnumpy(cp_einsum)
print("Einsum diff: {}".format(np_einsum - cp_einsum_cpu))

Installation

Source (pip install cupy)

Environment

OS                           : Linux-5.3.18-59.10-default-x86_64-with-glibc2.3.4
Python Version               : 3.6.15
CuPy Version                 : 9.6.0
CuPy Platform                : NVIDIA CUDA
NumPy Version                : 1.19.5
SciPy Version                : 1.5.4
Cython Build Version         : 0.29.26
Cython Runtime Version       : None
CUDA Root                    : /usr/local/cuda
nvcc PATH                    : /usr/local/cuda/bin/nvcc
CUDA Build Version           : 11040
CUDA Driver Version          : 11060
CUDA Runtime Version         : 11040
cuBLAS Version               : (available)
cuFFT Version                : 10502
cuRAND Version               : 10205
cuSOLVER Version             : (11, 2, 0)
cuSPARSE Version             : (available)
NVRTC Version                : (11, 4)
Thrust Version               : 101201
CUB Build Version            : 101201
Jitify Build Version         : <unknown>
cuDNN Build Version          : None
cuDNN Version                : None
NCCL Build Version           : None
NCCL Runtime Version         : None
cuTENSOR Version             : None
cuSPARSELt Build Version     : None
Device 0 Name                : NVIDIA GeForce RTX 3080 Ti
Device 0 Compute Capability  : 86
Device 0 PCI Bus ID          : 0000:01:00.0

Additional Information

No response

[emcastillo]

This is just something that we have to live with when using cupy?

yes, complex support is provided by nvidia libraries and due to the library/hardware internal details, CuPy can’t control this. Note that PyTorch provides the same result as CuPy

import numpy as np
import cupy as cp
import torch
num = np.random.rand() + 1j * np.random.rand()
num = np.array([num])
num_cp = cp.asarray(num)
print(num * num.conj())
print(num_cp * num_cp.conj())
num_torch = torch.tensor([num]).cuda()
print(num_torch * num_torch.conj())
------------------------------------
[0.73128476+0.j]
[0.73128476+2.85201533e-18j]
tensor([[0.7313-2.8520e-18j]], device='cuda:0', dtype=torch.complex128)

[RemingtonRohel]

Ahh okay, lots to learn here. Thank you both for the help!