BUG: wrong results in sparse.linalg.lobpcg

Describe your issue.

As first discussed in #10974 , I obtained wrong results when trying to approximate the SVD of a matrix with LOBPCG. ARPACK is fine.

The matrix can be found as a txt file at https://gist.github.com/ogauthe/63b70fd32329eac90150fa91dfb1ac67. It is pretty small with shape (504, 504), square but not symmetric, with ~78 non-zero singular values, without any degeneracy.

For k=1, everything is fine. For k=2, the result is completely wrong. For k>7, the functions crashes, so I suppose the problem comes from an undetected Cholesky failure, as in #10974.

ping @lobpcg

Reproducing Code Example

import numpy as np
import scipy.linalg as lg
import scipy.sparse.linalg as slg

A = np.loadtxt("lobpcg_fail.txt")
x = lg.norm(A)
print(A.shape, A.dtype)

k = 2
u0, s0, v0 = lg.svd(A)
u1, s1, v1 = slg.svds(A, k=k, solver="lobpcg")
u2, s2, v2 = slg.svds(A, k=k, solver="arpack")

print("dense  truncation", lg.norm(u0[:,:k] * s0[:k] @ v0[:k] - A) / x) 
print("arpack truncation", lg.norm(u2 * s2 @ v2 - A) / x) 
print("lobpcg truncation", lg.norm(u1 * s1 @ v1 - A) / x)

Error message

(504, 504) float64
dense  truncation 0.002749949291553053
arpack truncation 0.0027499492915530536
lobpcg truncation 1.2256989944723657

SciPy/NumPy/Python version information

1.6.2 1.20.3 sys.version_info(major=3, minor=7, micro=11, releaselevel=‘final’, serial=0)

[lobpcg]

After #14790 is merged, I plan to fix this issue as described just above in https://github.com/scipy/scipy/issues/14784#issuecomment-931712902

[lobpcg]

So I think that it is just the same issue as #10974 . It’s unclear how to fix it automatically in the lobpcg code. The algorithm of lobpcg is just not designed for such matrices with very high multiplicity of eigenvalues, especially zero. May be we should at least add a warning in the function docs.