Inverse of singular matrix. Should be error but in fact not.

Hello. I’ve got strange behavior for experiments. I’m working with matrix (for example b) that in result of multiplying b.T * b should be singular matrix and for inverse method should be arisen error like numpy.linalg.linalg.LinAlgError: Singular matrix. But result was high/low values. Code below:

>>> b = np.matrix([[1,1,0], [1,0,1], [1,1,0]])
>>> b
matrix([[1, 1, 0],
        [1, 0, 1],
        [1, 1, 0]])
>>> np.linalg.inv(b.T * b)
matrix([[ 4.50359963e+15, -4.50359963e+15, -4.50359963e+15],
        [-4.50359963e+15,  4.50359963e+15,  4.50359963e+15],
        [-4.50359963e+15,  4.50359963e+15,  4.50359963e+15]])

How can be avoided this behavior? Tests on: win10, Python 3.5.4, numpy version ‘1.14.0’. ubuntu 16.04, Python 3.5.2, numpy version ‘1.13.3’ and ‘1.14.0’.

PS. I’ve checked via wolfram and R it’s real singular matrix.

[njsmith]

The condition number (np.linalg.cond) might be a more appropriate tool than the determinant here.

[pmli]

@noob-saibot You could use scipy.linalg.solve, which raises a warning when it detects an ill-conditioned matrix (since version 0.19.0, if I’m not mistaken):

>>> import numpy as np
>>> import scipy.linalg as spla
>>> b = np.array([[1,1,0], [1,0,1], [1,1,0]])
>>> spla.solve(b.T.dot(b), np.eye(3))
/some/long/path/scipy/linalg/basic.py:40: RuntimeWarning: scipy.linalg.solve
Ill-conditioned matrix detected. Result is not guaranteed to be accurate.
Reciprocal condition number/precision: 1.2335811384723961e-17 / 1.1102230246251565e-16
  RuntimeWarning)
array([[ 4.50359963e+15, -4.50359963e+15, -4.50359963e+15],
       [-4.50359963e+15,  4.50359963e+15,  4.50359963e+15],
       [-4.50359963e+15,  4.50359963e+15,  4.50359963e+15]])

The issue in numpy.linalg.inv is that the LU decomposition doesn’t give the exact solution, which you can see from:

>>> P, L, U = spla.lu(b.T.dot(b))
>>> U
array([[ 3.00000000e+00,  2.00000000e+00,  1.00000000e+00],
       [ 0.00000000e+00,  6.66666667e-01, -6.66666667e-01],
       [ 0.00000000e+00,  0.00000000e+00,  2.22044605e-16]])

Since all diagonal elements in U are nonzero, numpy.linalg.inv continues without raising an error. If it used GESVX, as scipy.linalg.solve does, it could relatively cheaply compute an estimate of a condition number.

But, do you really need inv for what you are doing? If, e.g., you need to solve a least squares problem, there are better methods then using normal equations.