Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

cuFFT callback tests fail on sm_86

See original GitHub issue

On the latest master + sm_86, the failure happens at the linking stage.

Reproducer:

$ pytest tests/cupy_tests/fft_tests/test_callback.py -x
========================================================================= test session starts =========================================================================
platform linux -- Python 3.8.10, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /home/leofang/dev/cupy_cuda114, configfile: setup.cfg
collected 584 items                                                                                                                                                   

tests/cupy_tests/fft_tests/test_callback.py F

============================================================================== FAILURES ===============================================================================
_____________________________________________ Test1dCallbacks.test_fft_load[_param_0_{n=None, norm=None, shape=(10, 7)}] ______________________________________________

self = <<cupy_tests.fft_tests.test_callback.Test1dCallbacks object at 0x7f470e296a60>  parameter: {'n': None, 'norm': None, 'shape': (10, 7)}>
xp = <module 'cupy' from '/home/leofang/dev/cupy_cuda114/cupy/__init__.py'>, dtype = <class 'numpy.complex64'>

    @testing.for_complex_dtypes()
    @testing.numpy_cupy_allclose(rtol=1e-4, atol=1e-7, contiguous_check=False)
    def test_fft_load(self, xp, dtype):
>       return self._test_load_helper(xp, dtype, 'fft')

tests/cupy_tests/fft_tests/test_callback.py:146: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/cupy_tests/fft_tests/test_callback.py:138: in _test_load_helper
    with xp.fft.config.set_cufft_callbacks(cb_load=cb_load):
cupy/fft/_callback.pyx:567: in cupy.fft._callback.set_cufft_callbacks.__init__
    mgr = _CallbackManager(
cupy/fft/_callback.pyx:388: in cupy.fft._callback._CallbackManager.__init__
    _nvcc_link(tempdir, obj_host, obj_dev, arch, mod_filename,
cupy/fft/_callback.pyx:290: in cupy.fft._callback._nvcc_link
    p.check_returncode()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = CompletedProcess(args=['/usr/local/cuda-11.4/bin/nvcc', '-ccbin', 'g++', '-shared', '-arch=sm_86', '/tmp/tmpj_icd884/t...mpj_icd884/tmp8i5gc_9o/cupy_callback_a84d83fce0114cf19d18471ad9a57fd5.cpython-38-x86_64-linux-gnu.so'], returncode=255)

    def check_returncode(self):
        """Raise CalledProcessError if the exit code is non-zero."""
        if self.returncode:
>           raise CalledProcessError(self.returncode, self.args, self.stdout,
                                     self.stderr)
E           AssertionError: Only cupy raises error
E           
E           Traceback (most recent call last):
E             File "/home/leofang/dev/cupy_cuda114/cupy/testing/_helper.py", line 47, in _call_func
E               result = impl(self, *args, **kw)
E             File "/home/leofang/dev/cupy_cuda114/tests/cupy_tests/fft_tests/test_callback.py", line 146, in test_fft_load
E               return self._test_load_helper(xp, dtype, 'fft')
E             File "/home/leofang/dev/cupy_cuda114/tests/cupy_tests/fft_tests/test_callback.py", line 138, in _test_load_helper
E               with xp.fft.config.set_cufft_callbacks(cb_load=cb_load):
E             File "cupy/fft/_callback.pyx", line 567, in cupy.fft._callback.set_cufft_callbacks.__init__
E               mgr = _CallbackManager(
E             File "cupy/fft/_callback.pyx", line 388, in cupy.fft._callback._CallbackManager.__init__
E               _nvcc_link(tempdir, obj_host, obj_dev, arch, mod_filename,
E             File "cupy/fft/_callback.pyx", line 290, in cupy.fft._callback._nvcc_link
E               p.check_returncode()
E             File "/home/leofang/miniconda3/envs/cupy_cuda114_dev/lib/python3.8/subprocess.py", line 448, in check_returncode
E               raise CalledProcessError(self.returncode, self.args, self.stdout,
E           subprocess.CalledProcessError: Command '['/usr/local/cuda-11.4/bin/nvcc', '-ccbin', 'g++', '-shared', '-arch=sm_86', '/tmp/tmpj_icd884/tmp8i5gc_9o/cupy_callback_a84d83fce0114cf19d18471ad9a57fd5_dev.o', '/tmp/tmpj_icd884/tmp8i5gc_9o/cupy_callback_a84d83fce0114cf19d18471ad9a57fd5.o', '-L/tmp/tmpj_icd884', '-lcufft_static_10500_sm86', '-lculibos', '-lpthread', '-o', '/tmp/tmpj_icd884/tmp8i5gc_9o/cupy_callback_a84d83fce0114cf19d18471ad9a57fd5.cpython-38-x86_64-linux-gnu.so']' returned non-zero exit status 255.

../../miniconda3/envs/cupy_cuda114_dev/lib/python3.8/subprocess.py:448: AssertionError
------------------------------------------------------------------------ Captured stdout call -------------------------------------------------------------------------
dtype is <class 'numpy.complex64'>
------------------------------------------------------------------------ Captured stderr call -------------------------------------------------------------------------
nvlink error   : Undefined reference to 'lut_dp_2_13' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_17' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_19' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_23' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_29' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_31' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_37' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_41' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_43' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_47' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_53' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_59' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_61' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_67' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_71' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_73' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_79' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_83' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_89' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_97' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_101' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_103' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_107' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_109' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_113' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_127' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_8_32' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_8_64' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_8_256' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_8_512' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_16_64' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_16_128' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_16_256' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_16_1024' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_16_2048' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_16_4096' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_2_4' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_3_9' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_4_8' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_4_16' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_5_25' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_5_125' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_5_625' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_5_3125' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_6_36' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_6_216' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_6_1296' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_7_49' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_7_343' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_7_2401' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_8_2048' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_8_4096' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_9_27' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_9_81' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_9_243' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_9_729' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_9_2187' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_10_100' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_10_1000' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_11_121' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_11_1331' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_12_144' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_12_1728' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_32_256' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_dp_32_8192' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbdouble_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_13' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_17' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_19' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_23' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_29' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_31' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_37' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_41' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_43' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_47' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_53' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_59' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_61' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_67' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_71' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_73' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_79' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_83' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_89' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_97' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_101' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_103' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_107' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_109' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_113' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_127' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_prime_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_8_32' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_8_64' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_8_256' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_8_512' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_16_32' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_16_64' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_16_128' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_16_256' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_16_512' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_16_1024' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_16_2048' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_16_4096' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_16_8192' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_bluestein_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_2_4' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_3_9' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_4_8' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_4_16' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_5_25' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_5_125' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_5_625' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_5_3125' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_6_36' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_6_216' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_6_1296' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_7_49' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_7_343' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_7_2401' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_9_27' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_9_81' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_9_243' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_9_729' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_9_2187' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_10_100' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_10_1000' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_11_121' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_11_1331' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_12_144' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
nvlink error   : Undefined reference to 'lut_sp_12_1728' in '/tmp/tmpj_icd884/libcufft_static_10500_sm86.a:cbfloat_32bit_regular_callback_RT_PTX.o'
======================================================================= short test summary info =======================================================================
FAILED tests/cupy_tests/fft_tests/test_callback.py::Test1dCallbacks::test_fft_load[_param_0_{n=None, norm=None, shape=(10, 7)}] - AssertionError: Only cupy raises e...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================================================== 1 failed in 5.69s ==========================================================================

Environment:

>>> cp.show_config()
OS                           : Linux-5.4.0-72-generic-x86_64-with-glibc2.10
Python Version               : 3.8.10
CuPy Version                 : 10.0.0a2
CuPy Platform                : NVIDIA CUDA
NumPy Version                : 1.21.0
SciPy Version                : 1.7.0
Cython Build Version         : 0.29.24
Cython Runtime Version       : 0.29.24
CUDA Root                    : /usr/local/cuda-11.4
nvcc PATH                    : /usr/local/cuda-11.4/bin/nvcc
CUDA Build Version           : 11040
CUDA Driver Version          : 11040
CUDA Runtime Version         : 11040
cuBLAS Version               : (available)
cuFFT Version                : 10500
cuRAND Version               : 10205
cuSOLVER Version             : (11, 2, 0)
cuSPARSE Version             : (available)
NVRTC Version                : (11, 4)
Thrust Version               : 101201
CUB Build Version            : 101201
Jitify Build Version         : 60e9e72
cuDNN Build Version          : None
cuDNN Version                : None
NCCL Build Version           : 2507
NCCL Runtime Version         : 2507
cuTENSOR Version             : None
cuSPARSELt Build Version     : None
Device 0 Name                : NVIDIA RTX A6000
Device 0 Compute Capability  : 86
Device 0 PCI Bus ID          : 0000:4B:00.0

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:11 (11 by maintainers)

github_iconTop GitHub Comments

5reactions
mnicelycommented, Jul 15, 2021

Okay, we found the error. CuPy is pruning libcufft_static.a of any arch that is not the target.

sm80 and sm86 are binary compatible and both kinds of kernels use the same sm80 twiddles factors.

You shouldn’t remove the binary compatible symbols.

3reactions
leofangcommented, Jul 15, 2021

Verified that without pruning it works. Will send a patch later.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Simple cuFFT callback test not working
I have written a simple example to use the new cuFFT callback feature of CUDA 6.5, but it is not working. The full...
Read more >
cufft store callback called more than once - Stack Overflow
As I mentioned in the beginning, I see different behavior when using CUDA 9 instead of CUDA 8 for this test case (only...
Read more >
Support for CUFFT callbacks · Issue #75 · JuliaGPU/CUDA.jl
Input callbacks​​ CuFFT does not handle 8-bit data directly so without callbacks one has to pre-convert the input array to 32-bit floats.
Read more >
GPU Fast Convolution via the Overlap-and-Save Method in ...
The user interface we provide allows the user to test the ... The cuFFT callbacks allow the user a per-element access to the...
Read more >
1060 970 cuda - CSDN
When compiling with NVCC, the arch flag ('-arch') specifies the name of the NVIDIA GPU architecture that the CUDA files will be compiled...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

import cupy eats ~1.8Gb of virtual memory for nothing in every process instance

Next page

Cupy cannot be compiled depending on the combination of versions of Setuptools, Numpy, Cython