Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Support sparse matrices with more than 2^31-1 elements

See original GitHub issue

Currently there are a number of places in the codebase where the dtype for indptr and indices are hardcoded to 'i' (i.e., int32), for example: https://github.com/cupy/cupy/blob/22e270f303603848584974d1fbee5a38c9a5c428/cupyx/scipy/sparse/compressed.py#L330-L331 But they shouldn’t be. In SciPy, they use a helper function get_index_dtype() to determine this: https://github.com/scipy/scipy/blob/f4b5605031f738bc87ae4e193d614d525b98ffba/scipy/sparse/compressed.py#L65-L71 I think this change shouldn’t be hard, but it is worth a standalone PR with sufficient test coverage added.

For the short term, I’d say let’s encourage PR authors to take this into account, so we can make the changes gradually. In particular, when a new ElementwiseKernel or ReductionKernel is added, we just need to use template types T as oppose to hardcoded int32.

Issue Analytics

State:
Created 3 years ago
Reactions:1
Comments:8 (8 by maintainers)

Top GitHub Comments

2reactions

takagicommented, Dec 2, 2020

CuPy already has cuSPARSE Generic API supports, #3129 and #3242, awesome @anaruse!

1reaction

takagicommented, Aug 31, 2022

https://github.com/cupy/cupy/issues/3513#issuecomment-735208809 Fortunately, the oldest CTK that CuPy supports (10.2) has the Generic API now, so we don’t have to think of mixing it and older API (supports only 32bit indices).

Top Results From Across the Web

Sparse matrix support for long vectors (over 2^31 elements)

The issue is that the Matrix package in R cannot handle long vectors (length greater than 2^31 - 1). In my case, a...

Sparse Matrices - MATLAB & Simulink

Sparse matrices provide efficient storage of double or logical data that has a large percentage of zeros. While full (or dense) matrices store...

mfem::SparseMatrix Class Reference

SparseMatrix (int nrows, int ncols, int rowsize). Create a sparse matrix in CSR format where each row has space allocated for exactly rowsize...

Sparse Matrices — GSL 2.7 documentation - GNU.org

GSL currently supports three storage formats for sparse matrices: the ... to add more elements to the sparse matrix than the previously specified...

(PDF) Optimizing sparse matrix-multiple vectors multiplication for ...

Finally and most relevant to our focus in this paper, LOBPCG algorithm requires the multiplication of a sparse matrix with multiple vectors (SpMM), ......