RuntimeError: radix_sort: failed on 2nd step: cudaErrorInvalidValue: invalid argument

Upon investigating https://github.com/rapidsai/cuml/issues/4167 @Nanthini10 discovered that cupy seems to have an error:

This code: `import numpy as np

import cupy as cp

a = cp.array(np.arrange(2305))

a.argsort() `

produces the following error:

---

RuntimeError Traceback (most recent call last) <ipython-input-36-be7f84cea603> in <module> 1 import numpy as np 2 a = cp.array(np.arange(2305)) ----> 3 a.argsort()

cupy/_core/core.pyx in cupy._core.core.ndarray.argsort()

cupy/_core/core.pyx in cupy._core.core.ndarray.argsort()

cupy/_core/_routines_sorting.pyx in cupy._core._routines_sorting._ndarray_argsort()

cupy/cuda/thrust.pyx in cupy.cuda.thrust.argsort()

RuntimeError: radix_sort: failed on 2nd step: cudaErrorInvalidValue: invalid argument

However, it is important to note that using cp.asnumpy(…) works with no issue Additionally, the argument cp.array(np.arange(2304)) also works with no issue

Our environment is as follows: ’ Click here to see environment details git*

print_env.sh: 10: [: true: unexpected operator

Not inside a git repository

OS Information DISTRIB_ID=Ubuntu DISTRIB_RELEASE=20.04 DISTRIB_CODENAME=focal DISTRIB_DESCRIPTION=“Ubuntu 20.04.3 LTS” NAME=“Ubuntu” VERSION=“20.04.3 LTS (Focal Fossa)” ID=ubuntu ID_LIKE=debian PRETTY_NAME=“Ubuntu 20.04.3 LTS” VERSION_ID=“20.04” HOME_URL=“https://www.ubuntu.com/” SUPPORT_URL=“https://help.ubuntu.com/” BUG_REPORT_URL=“https://bugs.launchpad.net/ubuntu/” PRIVACY_POLICY_URL=“https://www.ubuntu.com/legal/terms-and-policies/privacy-policy” VERSION_CODENAME=focal UBUNTU_CODENAME=focal Linux bdf4ab8a676b 5.4.0-88-generic #99-Ubuntu SMP Thu Sep 23 17:29:00 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

GPU Information Tue Oct 5 22:00:45 2021 ±----------------------------------------------------------------------------+ | NVIDIA-SMI 460.91.03 Driver Version: 460.91.03 CUDA Version: 11.2 | |-------------------------------±---------------------±---------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 TITAN X (Pascal) On | 00000000:02:00.0 On | N/A | | 30% 49C P8 17W / 250W | 860MiB / 12173MiB | 8% Default | | | | N/A | ±------------------------------±---------------------±---------------------+ | 1 GeForce GTX 1080 On | 00000000:81:00.0 Off | N/A | | 0% 48C P8 6W / 180W | 11MiB / 8119MiB | 0% Default | | | | N/A | ±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| ±----------------------------------------------------------------------------+

CPU Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian Address sizes: 46 bits physical, 48 bits virtual CPU(s): 56 On-line CPU(s) list: 0-55 Thread(s) per core: 2 Core(s) per socket: 14 Socket(s): 2 NUMA node(s): 2 Vendor ID: GenuineIntel CPU family: 6 Model: 63 Model name: Genuine Intel® CPU @ 2.70GHz Stepping: 2 CPU MHz: 1197.298 CPU max MHz: 3500.0000 CPU min MHz: 1200.0000 BogoMIPS: 5387.46 Virtualization: VT-x L1d cache: 896 KiB L1i cache: 896 KiB L2 cache: 7 MiB L3 cache: 70 MiB NUMA node0 CPU(s): 0-13,28-41 NUMA node1 CPU(s): 14-27,42-55 Vulnerability Itlb multihit: KVM: Mitigation: Split huge pages Vulnerability L1tf: Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable Vulnerability Mds: Mitigation; Clear CPU buffers; SMT vulnerable Vulnerability Meltdown: Mitigation; PTI Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Full generic retpoline, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm cpuid_fault epb invpcid_single pti intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear flush_l1d

CMake /opt/conda/envs/rapids/bin/cmake cmake version 3.20.5

CMake suite maintained and supported by Kitware (kitware.com/cmake).

g++ /usr/local/bin/g++ g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 Copyright © 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

nvcc /usr/local/cuda/bin/nvcc nvcc: NVIDIA ® Cuda compiler driver Copyright © 2005-2021 NVIDIA Corporation Built on Sun_Feb_14_21:12:58_PST_2021 Cuda compilation tools, release 11.2, V11.2.152 Build cuda_11.2.r11.2/compiler.29618528_0

Python /opt/conda/envs/rapids/bin/python Python 3.8.12

Environment Variables PATH : /opt/conda/envs/rapids/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin LD_LIBRARY_PATH : /usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64:/usr/local/lib:/opt/conda/envs/rapids/lib NUMBAPRO_NVVM : NUMBAPRO_LIBDEVICE : CONDA_PREFIX : /opt/conda/envs/rapids PYTHON_PATH :

conda packages conda is /opt/conda/envs/rapids/bin/conda /opt/conda/envs/rapids/bin/conda

packages in environment at /opt/conda/envs/rapids:

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 1_llvm conda-forge abseil-cpp 20210324.2 h9c3ff4c_0 conda-forge aiobotocore 1.4.1 pyhd8ed1ab_0 conda-forge aiohttp 3.7.4.post0 py38h497a2fe_0 conda-forge aioitertools 0.8.0 pyhd8ed1ab_0 conda-forge alabaster 0.7.12 py_0 conda-forge alsa-lib 1.2.3 h516909a_0 conda-forge anyio 3.3.2 py38h578d9bd_0 conda-forge appdirs 1.4.4 pyh9f0ad1d_0 conda-forge argon2-cffi 20.1.0 py38h497a2fe_2 conda-forge arrow-cpp 5.0.0 py38hdc1b314_6_cuda conda-forge arrow-cpp-proc 3.0.0 cuda conda-forge asn1crypto 1.4.0 pyh9f0ad1d_0 conda-forge asvdb 0.4.2 g90e8f2c_40 rapidsai-nightly async-timeout 3.0.1 py_1000 conda-forge async_generator 1.10 py_0 conda-forge atk-1.0 2.36.0 h3371d22_4 conda-forge attrs 21.2.0 pyhd8ed1ab_0 conda-forge autoconf 2.69 pl5320h36c2ea0_10 conda-forge automake 1.16.2 pl5320ha770c72_3 conda-forge aws-c-cal 0.5.11 h95a6274_0 conda-forge aws-c-common 0.6.2 h7f98852_0 conda-forge aws-c-event-stream 0.2.7 h3541f99_13 conda-forge aws-c-io 0.10.5 hfb6a706_0 conda-forge aws-checksums 0.1.11 ha31a3da_7 conda-forge aws-sam-translator 1.38.0 pyhd8ed1ab_0 conda-forge aws-sdk-cpp 1.8.186 hb4091e7_3 conda-forge aws-xray-sdk 2.8.0 pyhd8ed1ab_0 conda-forge babel 2.9.1 pyh44b312d_0 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 py_2 conda-forge backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge beautifulsoup4 4.10.0 pyha770c72_0 conda-forge benchmark 1.5.1 he1b5a44_2 conda-forge black 19.10b0 py_4 conda-forge blas 2.111 openblas conda-forge blas-devel 3.9.0 11_linux64_openblas conda-forge blazingsql 21.10.0a0 pypi_0 pypi bleach 4.1.0 pyhd8ed1ab_0 conda-forge blinker 1.4 py_1 conda-forge blosc 1.21.0 h9c3ff4c_0 conda-forge bokeh 2.3.3 py38h578d9bd_0 conda-forge boost 1.72.0 py38h1e42940_1 conda-forge boost-cpp 1.72.0 h312852a_5 conda-forge boto3 1.17.106 pyhd8ed1ab_0 conda-forge botocore 1.20.106 pyhd8ed1ab_0 conda-forge brotli 1.0.9 pypi_0 pypi brotli-bin 1.0.9 h7f98852_5 conda-forge brotlipy 0.7.0 py38h497a2fe_1001 conda-forge brunsli 0.1 h9c3ff4c_0 conda-forge bsql-engine 0.6 pypi_0 pypi bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.17.2 h7f98852_0 conda-forge ca-certificates 2021.5.30 ha878542_0 conda-forge cachetools 4.2.4 pyhd8ed1ab_0 conda-forge cairo 1.16.0 h6cf1ce9_1008 conda-forge certifi 2021.5.30 py38h578d9bd_0 conda-forge cffi 1.14.6 py38h3931269_1 conda-forge cfitsio 3.470 hb418390_7 conda-forge cfn-lint 0.54.2 py38h578d9bd_0 conda-forge chardet 4.0.0 py38h578d9bd_1 conda-forge charls 2.2.0 h9c3ff4c_0 conda-forge charset-normalizer 2.0.0 pyhd8ed1ab_0 conda-forge clang 11.0.0 default_h934c63c_0 conda-forge clang-tools 11.0.0 default_h934c63c_0 conda-forge clangxx 11.0.0 default_hde54327_0 conda-forge click 7.1.2 pyh9f0ad1d_0 conda-forge click-plugins 1.1.1 py_0 conda-forge cligj 0.7.2 pyhd8ed1ab_0 conda-forge cloudpickle 2.0.0 pyhd8ed1ab_0 conda-forge cmake 3.20.5 h8897547_0 conda-forge cmake-format 0.6.11 pyh9f0ad1d_0 conda-forge cmake_setuptools 0.1.3 py_0 rapidsai-nightly cmarkgfm 0.6.0 py38h497a2fe_0 conda-forge colorama 0.4.4 pyh9f0ad1d_0 conda-forge colorcet 2.0.6 pyhd8ed1ab_0 conda-forge commonmark 0.9.1 py_0 conda-forge conda 4.10.3 py38h578d9bd_2 conda-forge conda-build 3.20.3 py38h32f6830_0 conda-forge conda-package-handling 1.7.3 py38h497a2fe_0 conda-forge conda-verify 3.1.1 py38h578d9bd_1003 conda-forge coverage 5.5 py38h497a2fe_0 conda-forge cppzmq 4.8.1 hf7cf922_0 conda-forge cryptography 3.4.7 py38ha5dfef3_0 conda-forge cudatoolkit 11.2.72 h2bc3f7f_0 nvidia cudf 0+untagged.1.gfc341af pypi_0 pypi cudf-kafka 0+untagged.1.gfc341af pypi_0 pypi cugraph 0+untagged.1.g54b8573 pypi_0 pypi cuml 0+untagged.1.g835a9ae pypi_0 pypi cupy 9.4.0 py38h7818112_0 conda-forge curl 7.79.1 h2574ce0_1 conda-forge cusignal 0+untagged.1.g5ae8e6a pypi_0 pypi cuspatial 0+untagged.1.gb047251 pypi_0 pypi cuxfilter 0+untagged.1.ge7ef0f0 pypi_0 pypi cycler 0.10.0 py_2 conda-forge cyrus-sasl 2.1.27 h230043b_3 conda-forge cython 0.29.24 py38h709712a_0 conda-forge cytoolz 0.11.0 py38h497a2fe_3 conda-forge dash 2.0.0 pypi_0 pypi dash-bootstrap-components 0.13.1 pypi_0 pypi dash-core-components 2.0.0 pypi_0 pypi dash-html-components 2.0.0 pypi_0 pypi dash-table 5.0.0 pypi_0 pypi dask 2021.9.1 pyhd8ed1ab_0 conda-forge dask-core 2021.9.1 pyhd8ed1ab_0 conda-forge dask-cudf 0+untagged.1.gfc341af pypi_0 pypi dask-glm 0.2.0 py_1 conda-forge dask-labextension 5.1.0 pyhd8ed1ab_0 conda-forge dask-ml 1.9.0 pyhd8ed1ab_0 conda-forge dataclasses 0.8 pyhc8e2a94_3 conda-forge datashader 0.11.1 pyh9f0ad1d_0 conda-forge datashape 0.5.4 py_1 conda-forge dbus 1.13.6 h48d8840_2 conda-forge decorator 4.4.2 py_0 conda-forge defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge distributed 2021.9.1 py38h578d9bd_0 conda-forge dlpack 0.5 h9c3ff4c_0 conda-forge docker-py 5.0.2 py38h578d9bd_0 conda-forge docker-pycreds 0.4.0 py_0 conda-forge docutils 0.16 py38h578d9bd_3 conda-forge double-conversion 3.1.5 h9c3ff4c_2 conda-forge doxygen 1.8.20 had0d8f1_0 conda-forge ecdsa 0.17.0 pyhd8ed1ab_0 conda-forge entrypoints 0.3 pyhd8ed1ab_1003 conda-forge execnet 1.9.0 pyhd8ed1ab_0 conda-forge expat 2.4.1 h9c3ff4c_0 conda-forge fa2 0.3.5 py38h1e0a361_0 conda-forge faiss-proc 1.0.0 cuda rapidsai fastavro 1.4.5 py38h497a2fe_0 conda-forge fastrlock 0.6 py38h709712a_1 conda-forge feather-format 0.4.1 pyh9f0ad1d_0 conda-forge filelock 3.1.0 pyhd8ed1ab_1 conda-forge filterpy 1.4.5 py_1 conda-forge fiona 1.8.20 py38hbb147eb_1 conda-forge flake8 3.8.4 py_0 conda-forge flask 2.0.1 pyhd8ed1ab_0 conda-forge flask-compress 1.10.1 pypi_0 pypi flask_cors 3.0.10 pyhd3deb0d_0 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.13.1 hba837de_1005 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge freetype 2.10.4 h0708190_1 conda-forge freexl 1.0.6 h7f98852_0 conda-forge fribidi 1.0.10 h36c2ea0_0 conda-forge fsspec 2021.9.0 pyhd8ed1ab_0 conda-forge future 0.18.2 py38h578d9bd_3 conda-forge gcsfs 2021.9.0 pyhd8ed1ab_0 conda-forge gdal 3.3.1 py38h81a01a0_3 conda-forge gdk-pixbuf 2.42.6 h04a7f16_0 conda-forge geopandas 0.9.0 pyhd8ed1ab_1 conda-forge geopandas-base 0.9.0 pyhd8ed1ab_1 conda-forge geos 3.9.1 h9c3ff4c_2 conda-forge geotiff 1.6.0 h4f31c25_6 conda-forge gettext 0.19.8.1 h73d1719_1008 conda-forge gflags 2.2.2 he1b5a44_1004 conda-forge giflib 5.2.1 h36c2ea0_2 conda-forge git 2.33.0 pl5321hc30692c_1 conda-forge git-lfs 2.13.3 ha770c72_0 conda-forge glib 2.68.4 h9c3ff4c_1 conda-forge glib-tools 2.68.4 h9c3ff4c_1 conda-forge glob2 0.7 py_0 conda-forge glog 0.5.0 h48cff8f_0 conda-forge gmock 1.10.0 h4bd325d_7 conda-forge gmp 6.2.1 h58526e2_0 conda-forge google-auth 2.2.1 pyh6c4a22f_0 conda-forge google-auth-oauthlib 0.4.6 pyhd8ed1ab_0 conda-forge google-cloud-cpp 1.31.1 h3a30730_1 conda-forge gpuci-tools 0.3.1 10 gpuci graphite2 1.3.13 h58526e2_1001 conda-forge graphviz 2.49.0 h85b4f2f_0 conda-forge greenlet 1.1.2 py38h709712a_0 conda-forge grpc-cpp 1.40.0 h850795e_0 conda-forge gtest 1.10.0 h4bd325d_7 conda-forge gtk2 2.24.33 h539f30e_1 conda-forge gts 0.7.6 h64030ff_2 conda-forge harfbuzz 2.9.1 h83ec7ef_1 conda-forge hdbscan 0.8.27 py38h5c078b8_0 conda-forge hdf4 4.2.15 h10796ff_3 conda-forge hdf5 1.12.1 nompi_h2750804_100 conda-forge heapdict 1.0.1 py_0 conda-forge holoviews 1.14.6 pyhd8ed1ab_0 conda-forge html5lib 1.1 pyh9f0ad1d_0 conda-forge httpretty 1.1.4 pyhd8ed1ab_0 conda-forge huggingface_hub 0.0.17 pyhd8ed1ab_0 conda-forge hypothesis 6.23.1 pyhd8ed1ab_0 conda-forge icu 68.1 h58526e2_0 conda-forge idna 2.10 pyh9f0ad1d_0 conda-forge imagecodecs 2021.7.30 py38hb5ce8f7_1 conda-forge imageio 2.9.0 py_0 conda-forge imagesize 1.2.0 py_0 conda-forge importlib-metadata 4.8.1 py38h578d9bd_0 conda-forge importlib_metadata 4.8.1 hd8ed1ab_0 conda-forge inflection 0.5.1 pyh9f0ad1d_0 conda-forge iniconfig 1.1.1 pyh9f0ad1d_0 conda-forge ipykernel 5.5.5 py38hd0cf306_0 conda-forge ipython 7.15.0 py38h32f6830_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge ipywidgets 7.6.5 pyhd8ed1ab_0 conda-forge isort 5.6.4 py_0 conda-forge itsdangerous 2.0.1 pyhd8ed1ab_0 conda-forge jbig 2.1 h7f98852_2003 conda-forge jedi 0.17.2 py38h578d9bd_1 conda-forge jeepney 0.7.1 pyhd8ed1ab_0 conda-forge jinja2 3.0.1 pyhd8ed1ab_0 conda-forge jmespath 0.10.0 pyh9f0ad1d_0 conda-forge joblib 1.0.1 pyhd8ed1ab_0 conda-forge jpeg 9d h36c2ea0_0 conda-forge jpype1 1.3.0 py38h1fd1430_0 conda-forge json-c 0.15 h98cffda_0 conda-forge json5 0.9.5 pyh9f0ad1d_0 conda-forge jsondiff 1.3.0 pyhd8ed1ab_0 conda-forge jsonpatch 1.32 pyhd8ed1ab_0 conda-forge jsonpointer 2.0 py_0 conda-forge jsonschema 3.2.0 pyhd8ed1ab_3 conda-forge junit-xml 1.9 pyh9f0ad1d_0 conda-forge jupyter-packaging 0.7.12 pyhd8ed1ab_0 conda-forge jupyter-server-proxy 3.1.0 pyhd8ed1ab_0 conda-forge jupyter_client 7.0.5 pyhd8ed1ab_0 conda-forge jupyter_core 4.8.1 py38h578d9bd_0 conda-forge jupyter_server 1.11.0 pyhd8ed1ab_0 conda-forge jupyter_sphinx 0.3.1 py38h578d9bd_1 conda-forge jupyterlab 3.1.14 pyhd8ed1ab_0 conda-forge jupyterlab-favorites 3.0.0 pyhd8ed1ab_0 conda-forge jupyterlab-nvdashboard 0.7.0a210915 py_7 rapidsai-nightly jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge jupyterlab_server 2.8.2 pyhd8ed1ab_0 conda-forge jupyterlab_widgets 1.0.2 pyhd8ed1ab_0 conda-forge jxrlib 1.1 h7f98852_2 conda-forge kealib 1.4.14 h87e4c3c_3 conda-forge keyring 23.2.1 py38h578d9bd_0 conda-forge kiwisolver 1.3.2 py38h1fd1430_0 conda-forge krb5 1.19.2 hcc1bbae_2 conda-forge lapack 3.9.0 netlib conda-forge lcms2 2.12 hddcbb42_0 conda-forge ld_impl_linux-64 2.36.1 hea4e1c9_2 conda-forge lerc 2.2.1 h9c3ff4c_0 conda-forge libaec 1.0.6 h9c3ff4c_0 conda-forge libarchive 3.5.2 hccf745f_1 conda-forge libblas 3.9.0 11_linux64_openblas conda-forge libbrotlicommon 1.0.9 h7f98852_5 conda-forge libbrotlidec 1.0.9 h7f98852_5 conda-forge libbrotlienc 1.0.9 h7f98852_5 conda-forge libcblas 3.9.0 11_linux64_openblas conda-forge libclang-cpp11 11.0.0 default_ha5c780c_2 conda-forge libcrc32c 1.1.1 h9c3ff4c_2 conda-forge libcumlprims 21.10.00a210927 cuda11.2_g8e4d5a6_6 rapidsai-nightly libcurl 7.79.1 h2574ce0_1 conda-forge libcypher-parser 0.6.2 1 rapidsai-nightly libdap4 3.20.6 hd7c4107_2 conda-forge libdeflate 1.7 h7f98852_5 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libevent 2.1.10 hcdb4288_3 conda-forge libfaiss 1.7.0 cuda112h5bea7ad_8_cuda conda-forge libffi 3.4.2 h9c3ff4c_4 conda-forge libgcc-ng 9.4.0 hfa6338b_9 conda-forge libgcrypt 1.9.4 h7f98852_0 conda-forge libgd 2.3.3 h6ad9fb6_0 conda-forge libgdal 3.3.1 h6214c1d_3 conda-forge libgfortran-ng 9.4.0 h69a702a_9 conda-forge libgfortran5 9.4.0 h62347ff_9 conda-forge libglib 2.68.4 h174f98d_1 conda-forge libgpg-error 1.42 h9c3ff4c_0 conda-forge libgsasl 1.10.0 h5b4c23d_0 conda-forge libhwloc 2.3.0 h5e5b7d1_1 conda-forge libiconv 1.16 h516909a_0 conda-forge libkml 1.3.0 hd79254b_1012 conda-forge liblapack 3.9.0 11_linux64_openblas conda-forge liblapacke 3.9.0 11_linux64_openblas conda-forge liblief 0.11.5 h9c3ff4c_0 conda-forge libllvm10 10.0.1 he513fc3_3 conda-forge libllvm11 11.0.1 hf817b99_0 conda-forge libnetcdf 4.8.1 nompi_hb3fd0d9_101 conda-forge libnghttp2 1.43.0 h812cca2_1 conda-forge libntlm 1.4 h7f98852_1002 conda-forge libopenblas 0.3.17 pthreads_h8fe5266_1 conda-forge libpng 1.6.37 h21135ba_2 conda-forge libpq 13.3 hd57d9b9_0 conda-forge libprotobuf 3.16.0 h780b84a_0 conda-forge librdkafka 1.6.1 hc49e61c_1 conda-forge librsvg 2.52.0 hc3c00ef_0 conda-forge librttopo 1.1.0 h1185371_6 conda-forge libsodium 1.0.18 h36c2ea0_1 conda-forge libspatialindex 1.9.3 h9c3ff4c_4 conda-forge libspatialite 5.0.1 h8694cbe_6 conda-forge libssh2 1.10.0 ha56f1ee_2 conda-forge libstdcxx-ng 9.4.0 h79bfe98_9 conda-forge libthrift 0.15.0 he6d91bd_0 conda-forge libtiff 4.3.0 hf544144_1 conda-forge libtool 2.4.6 h9c3ff4c_1008 conda-forge libutf8proc 2.6.1 h7f98852_0 conda-forge libuuid 2.32.1 h7f98852_1000 conda-forge libuv 1.42.0 h7f98852_0 conda-forge libwebp 1.2.1 h3452ae3_0 conda-forge libwebp-base 1.2.1 h7f98852_0 conda-forge libxcb 1.13 h7f98852_1003 conda-forge libxml2 2.9.12 h72842e0_0 conda-forge libxslt 1.1.33 h15afd5d_2 conda-forge libzip 1.8.0 h4de3113_1 conda-forge libzopfli 1.0.3 h9c3ff4c_0 conda-forge lightgbm 3.2.1 py38h709712a_0 conda-forge llvm-openmp 12.0.1 h4bd325d_1 conda-forge llvmlite 0.36.0 py38h4630a5e_0 conda-forge locket 0.2.1 pypi_0 pypi lxml 4.6.3 py38hf1fe3a4_0 conda-forge lz4-c 1.9.3 h9c3ff4c_1 conda-forge lzo 2.10 h516909a_1000 conda-forge m4 1.4.18 h516909a_1001 conda-forge make 4.3 hd18ef5c_1 conda-forge mapclassify 2.4.3 pyhd8ed1ab_0 conda-forge markdown 3.3.4 pyhd8ed1ab_0 conda-forge markupsafe 2.0.1 py38h497a2fe_0 conda-forge matplotlib-base 3.4.3 py38hf4fb855_1 conda-forge maven 3.6.3 ha770c72_0 conda-forge mccabe 0.6.1 py_1 conda-forge mimesis 4.0.0 pyh9f0ad1d_0 conda-forge mistune 0.8.4 py38h497a2fe_1004 conda-forge mock 4.0.3 py38h578d9bd_1 conda-forge more-itertools 8.10.0 pyhd8ed1ab_0 conda-forge moto 2.2.4 pyhd8ed1ab_0 conda-forge msgpack-python 1.0.2 py38h1fd1430_1 conda-forge multidict 5.1.0 py38h497a2fe_1 conda-forge multipledispatch 0.6.0 py_0 conda-forge munch 2.5.0 py_0 conda-forge mypy 0.782 py_0 conda-forge mypy_extensions 0.4.3 py38h578d9bd_3 conda-forge mysql-connector-cpp 8.0.23 h812cca2_0 conda-forge nbclassic 0.3.2 pyhd8ed1ab_0 conda-forge nbclient 0.5.4 pyhd8ed1ab_0 conda-forge nbconvert 6.2.0 py38h578d9bd_0 conda-forge nbformat 5.1.3 pyhd8ed1ab_0 conda-forge nbsphinx 0.8.7 pyhd8ed1ab_0 conda-forge nccl 2.10.3.1 hdc17891_0 conda-forge ncurses 6.2 h58526e2_4 conda-forge nest-asyncio 1.5.1 pyhd8ed1ab_0 conda-forge netifaces 0.10.9 py38h497a2fe_1003 conda-forge networkx 2.6.3 pyhd8ed1ab_0 conda-forge ninja 1.10.2 h4bd325d_1 conda-forge nlohmann_json 3.9.1 h9c3ff4c_1 conda-forge nltk 3.6.3 pyhd8ed1ab_0 conda-forge nodejs 14.17.4 h92b4a50_0 conda-forge notebook 6.4.4 pyha770c72_0 conda-forge numba 0.53.1 py38h8b71fd7_1 conda-forge numpy 1.21.2 py38he2449b9_0 conda-forge numpydoc 1.1.0 py_1 conda-forge nvtx 0.2.3 py38h497a2fe_0 conda-forge oauthlib 3.1.1 pyhd8ed1ab_0 conda-forge olefile 0.46 pyh9f0ad1d_1 conda-forge openblas 0.3.17 pthreads_h4748800_1 conda-forge openjdk 11.0.9.1 h5cc2fde_1 conda-forge openjpeg 2.4.0 hb52868f_1 conda-forge openslide 3.4.1 h978ee9a_4 conda-forge openssl 1.1.1l h7f98852_0 conda-forge orc 1.6.10 h58a87f1_0 conda-forge packaging 21.0 pyhd8ed1ab_0 conda-forge pandas 1.3.3 py38h43a58ef_0 conda-forge pandoc 1.19.2 0 conda-forge pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge panel 0.12.1 pyhd8ed1ab_0 conda-forge pango 1.48.10 hb8ff022_1 conda-forge param 1.11.1 pyh6c4a22f_0 conda-forge parquet-cpp 1.5.1 2 conda-forge parso 0.7.1 pyh9f0ad1d_0 conda-forge partd 1.2.0 pyhd8ed1ab_0 conda-forge patchelf 0.13 h58526e2_0 conda-forge pathspec 0.9.0 pyhd8ed1ab_0 conda-forge patsy 0.5.2 pyhd8ed1ab_0 conda-forge pcre 8.45 h9c3ff4c_0 conda-forge pcre2 10.37 h032f7d1_0 conda-forge perl 5.32.1 0_h7f98852_perl5 conda-forge pexpect 4.8.0 pyh9f0ad1d_2 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 8.3.2 py38h8e6f84c_0 conda-forge pip 21.2.4 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h36c2ea0_0 conda-forge pkg-config 0.29.2 h36c2ea0_1008 conda-forge pkginfo 1.7.1 pyhd8ed1ab_0 conda-forge plotly 5.3.1 pypi_0 pypi pluggy 1.0.0 py38h578d9bd_1 conda-forge pooch 1.5.1 pyhd8ed1ab_0 conda-forge poppler 21.03.0 h93df280_0 conda-forge poppler-data 0.4.11 hd8ed1ab_0 conda-forge postgresql 13.3 h2510834_0 conda-forge proj 8.0.1 h277dcde_0 conda-forge prometheus_client 0.11.0 pyhd8ed1ab_0 conda-forge prompt-toolkit 3.0.20 pyha770c72_0 conda-forge prompt_toolkit 3.0.20 hd8ed1ab_0 conda-forge protobuf 3.16.0 py38h709712a_0 conda-forge psutil 5.8.0 py38h497a2fe_1 conda-forge pthread-stubs 0.4 h36c2ea0_1001 conda-forge ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge pure-sasl 0.6.2 pyhd8ed1ab_0 conda-forge py 1.10.0 pyhd3deb0d_0 conda-forge py-cpuinfo 8.0.0 pyhd8ed1ab_0 conda-forge py-lief 0.11.5 py38h709712a_0 conda-forge pyarrow 5.0.0 py38hed47224_6_cuda conda-forge pyasn1 0.4.8 py_0 conda-forge pyasn1-modules 0.2.7 py_0 conda-forge pycodestyle 2.6.0 pyh9f0ad1d_0 conda-forge pycosat 0.6.3 py38h497a2fe_1006 conda-forge pycparser 2.20 pyh9f0ad1d_2 conda-forge pyct 0.4.6 py_0 conda-forge pyct-core 0.4.6 py_0 conda-forge pydata-sphinx-theme 0.6.3 pyhd8ed1ab_0 conda-forge pydeck 0.5.0 pyh9f0ad1d_0 conda-forge pydocstyle 6.1.1 pyhd8ed1ab_0 conda-forge pyee 8.1.0 pyh9f0ad1d_0 conda-forge pyflakes 2.2.0 pyh9f0ad1d_0 conda-forge pygal 3.0.0.dev1 pypi_0 pypi pygments 2.10.0 pyhd8ed1ab_0 conda-forge pyhive 0.6.4 pyhd8ed1ab_0 conda-forge pyjwt 2.1.0 pyhd8ed1ab_0 conda-forge pylibcugraph 0+untagged.1.g54b8573 pypi_0 pypi pynndescent 0.5.4 pyh6c4a22f_0 conda-forge pynvml 11.0.0 pyhd8ed1ab_0 conda-forge pyopenssl 20.0.1 pyhd8ed1ab_0 conda-forge pyorc 0.4.0 py38haf60add_5 conda-forge pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge pyppeteer 0.2.6 pyhd8ed1ab_0 conda-forge pyproj 3.1.0 py38h4df08a6_4 conda-forge pyrsistent 0.17.3 py38h497a2fe_2 conda-forge pysocks 1.7.1 py38h578d9bd_3 conda-forge pytest 6.2.5 py38h578d9bd_0 conda-forge pytest-asyncio 0.12.0 py38h32f6830_2 conda-forge pytest-benchmark 3.4.1 pyhd8ed1ab_0 conda-forge pytest-cov 2.12.1 pyhd8ed1ab_0 conda-forge pytest-forked 1.3.0 pyhd3deb0d_0 conda-forge pytest-timeout 1.4.2 pyh9f0ad1d_0 conda-forge pytest-xdist 2.4.0 pyhd8ed1ab_0 conda-forge python 3.8.12 hb7a2778_1_cpython conda-forge python-confluent-kafka 1.6.0 py38h497a2fe_1 conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-jose 3.3.0 pyh6c4a22f_1 conda-forge python-libarchive-c 3.1 py38h578d9bd_0 conda-forge python-louvain 0.15 pyhd3deb0d_0 conda-forge python-snappy 0.6.0 py38h49bdff1_0 conda-forge python_abi 3.8 2_cp38 conda-forge pytz 2021.1 pyhd8ed1ab_0 conda-forge pyu2f 0.1.5 pyhd8ed1ab_0 conda-forge pyviz_comms 2.1.0 pyhd8ed1ab_0 conda-forge pywavelets 1.1.1 py38h6c62de6_3 conda-forge pyyaml 5.4.1 py38h497a2fe_1 conda-forge pyzmq 22.3.0 py38h2035c66_0 conda-forge rapidjson 1.1.0 he1b5a44_1002 conda-forge re2 2021.09.01 h9c3ff4c_0 conda-forge readline 8.1 h46c0cb4_0 conda-forge readme_renderer 27.0 pyh9f0ad1d_0 conda-forge recommonmark 0.7.1 pyhd8ed1ab_0 conda-forge regex 2021.9.30 py38h497a2fe_0 conda-forge requests 2.26.0 pyhd8ed1ab_0 conda-forge requests-oauthlib 1.3.0 pyh9f0ad1d_0 conda-forge requests-toolbelt 0.9.1 py_0 conda-forge requests-unixsocket 0.2.0 py_0 conda-forge responses 0.14.0 pyhd8ed1ab_0 conda-forge rfc3986 1.5.0 pyhd8ed1ab_0 conda-forge rhash 1.4.1 h7f98852_0 conda-forge ripgrep 13.0.0 habb4d0f_1 conda-forge rmm 0+untagged.1.g12f82fb pypi_0 pypi rsa 4.7.2 pyh44b312d_0 conda-forge rtree 0.9.7 py38h02d302b_2 conda-forge ruamel_yaml 0.15.80 py38h497a2fe_1004 conda-forge s2n 1.0.10 h9b69904_0 conda-forge s3fs 2021.9.0 pyhd8ed1ab_1 conda-forge s3transfer 0.4.2 pyhd8ed1ab_0 conda-forge sacremoses 0.0.43 pyh9f0ad1d_0 conda-forge scikit-image 0.18.3 py38h43a58ef_0 conda-forge scikit-learn 0.24.2 py38hacb3eff_1 conda-forge scipy 1.6.0 py38hb2138dd_0 conda-forge seaborn 0.11.2 hd8ed1ab_0 conda-forge seaborn-base 0.11.2 pyhd8ed1ab_0 conda-forge secretstorage 3.3.1 py38h578d9bd_0 conda-forge send2trash 1.8.0 pyhd8ed1ab_0 conda-forge setuptools 49.6.0 py38h578d9bd_3 conda-forge shapely 1.7.1 py38hb7fe4a8_5 conda-forge shellcheck 0.7.2 ha770c72_1 conda-forge simpervisor 0.4 pyhd8ed1ab_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge snappy 1.1.8 he1b5a44_3 conda-forge sniffio 1.2.0 py38h578d9bd_1 conda-forge snowballstemmer 2.1.0 pyhd8ed1ab_0 conda-forge sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge soupsieve 2.0.1 py_1 conda-forge spdlog 1.8.5 h4bd325d_0 conda-forge sphinx 4.2.0 pyh6c4a22f_0 conda-forge sphinx-click 3.0.1 pyhd8ed1ab_0 conda-forge sphinx-copybutton 0.4.0 pyhd8ed1ab_0 conda-forge sphinx-markdown-tables 0.0.15 pyhd3deb0d_0 conda-forge sphinx_rtd_theme 1.0.0 pyhd8ed1ab_0 conda-forge sphinxcontrib-applehelp 1.0.2 py_0 conda-forge sphinxcontrib-devhelp 1.0.2 py_0 conda-forge sphinxcontrib-htmlhelp 2.0.0 pyhd8ed1ab_0 conda-forge sphinxcontrib-jsmath 1.0.1 py_0 conda-forge sphinxcontrib-qthelp 1.0.3 py_0 conda-forge sphinxcontrib-serializinghtml 1.1.5 pyhd8ed1ab_0 conda-forge sphinxcontrib-websupport 1.2.4 pyh9f0ad1d_0 conda-forge sqlalchemy 1.4.25 py38h497a2fe_0 conda-forge sqlite 3.36.0 h9cd32fc_2 conda-forge sshpubkeys 3.1.0 py_0 conda-forge statsmodels 0.12.2 py38h6c62de6_0 conda-forge streamz 0.6.2 pyh44b312d_0 conda-forge tbb 2020.2 h4bd325d_4 conda-forge tblib 1.7.0 pyhd8ed1ab_0 conda-forge tenacity 8.0.1 pypi_0 pypi terminado 0.12.1 py38h578d9bd_0 conda-forge testpath 0.5.0 pyhd8ed1ab_0 conda-forge threadpoolctl 2.2.0 pyh8a188c0_0 conda-forge thrift 0.14.0 py38h709712a_0 conda-forge thrift_sasl 0.4.3 pyhd8ed1ab_1 conda-forge tifffile 2021.8.30 pyhd8ed1ab_0 conda-forge tiledb 2.3.4 he87e0bf_0 conda-forge tk 8.6.11 h27826a3_1 conda-forge tokenizers 0.10.3 py38hb63a372_1 conda-forge toml 0.10.2 pyhd8ed1ab_0 conda-forge toolz 0.11.1 py_0 conda-forge tornado 6.1 py38h497a2fe_1 conda-forge tqdm 4.62.3 pyhd8ed1ab_0 conda-forge traitlets 5.1.0 pyhd8ed1ab_0 conda-forge transformers 4.6.1 pyhd8ed1ab_0 conda-forge treelite 2.1.0 py38hdd725b4_0 conda-forge treelite-runtime 2.1.0 pypi_0 pypi twine 3.4.2 pyhd8ed1ab_0 conda-forge typed-ast 1.4.3 py38h497a2fe_0 conda-forge typing-extensions 3.10.0.2 hd8ed1ab_0 conda-forge typing_extensions 3.10.0.2 pyha770c72_0 conda-forge tzcode 2021a h7f98852_2 conda-forge tzdata 2021a he74cb21_1 conda-forge ucx 1.11.1+gc58db6b cuda11.2_0 rapidsai-nightly ucx-proc 1.0.0 gpu rapidsai-nightly ucx-py 0.22.0a210930 py38_gc58db6b_24 rapidsai-nightly umap-learn 0.5.1 py38h578d9bd_1 conda-forge urllib3 1.26.7 pyhd8ed1ab_0 conda-forge wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge webencodings 0.5.1 py_1 conda-forge websocket-client 0.57.0 py38h578d9bd_4 conda-forge websockets 9.1 py38h497a2fe_0 conda-forge werkzeug 2.0.1 pyhd8ed1ab_0 conda-forge wheel 0.37.0 pyhd8ed1ab_1 conda-forge widgetsnbextension 3.5.1 py38h578d9bd_4 conda-forge wrapt 1.12.1 py38h497a2fe_3 conda-forge xarray 0.19.0 pyhd8ed1ab_1 conda-forge xerces-c 3.2.3 h9d8b166_2 conda-forge xgboost 1.4.2 pypi_0 pypi xmltodict 0.12.0 py_0 conda-forge xorg-fixesproto 5.0 h7f98852_1002 conda-forge xorg-inputproto 2.3.2 h7f98852_1002 conda-forge xorg-kbproto 1.0.7 h7f98852_1002 conda-forge xorg-libice 1.0.10 h7f98852_0 conda-forge xorg-libsm 1.2.3 hd9c2040_1000 conda-forge xorg-libx11 1.7.2 h7f98852_0 conda-forge xorg-libxau 1.0.9 h7f98852_0 conda-forge xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge xorg-libxext 1.3.4 h7f98852_1 conda-forge xorg-libxfixes 5.0.3 h7f98852_1004 conda-forge xorg-libxi 1.7.10 h7f98852_0 conda-forge xorg-libxrender 0.9.10 h7f98852_1003 conda-forge xorg-libxtst 1.2.3 h7f98852_1002 conda-forge xorg-recordproto 1.14.2 h7f98852_1002 conda-forge xorg-renderproto 0.11.1 h7f98852_1002 conda-forge xorg-xextproto 7.3.0 h7f98852_1002 conda-forge xorg-xproto 7.0.31 h7f98852_1007 conda-forge xz 5.2.5 h516909a_1 conda-forge yaml 0.2.5 h516909a_0 conda-forge yarl 1.6.3 py38h497a2fe_2 conda-forge zeromq 4.3.4 h9c3ff4c_1 conda-forge zfp 0.5.5 h9c3ff4c_7 conda-forge zict 2.0.0 pypi_0 pypi zipp 3.5.0 pyhd8ed1ab_0 conda-forge zlib 1.2.11 h36c2ea0_1011 conda-forge zstd 1.5.0 ha95c52a_0 conda-forge’

[robomike]

@leofang @Nanthini10

No GTX 1080 cards to test? There is no availability for newer cards yet, and the GTX 1080 is specified as the minimum requirement for most Nvidia products like issac sim

[Nanthini10]

@leofang I’ve tried to reproduce on different GPU cards wondering if this was an architecture based failure - didn’t fail on P100, V100 or RTX 8000.