Gauss Newton Minimization Error

I was working through the SimPEG examples trying to understand how the guts of the code work and I find the easiest way to do that is to keep it as simple as possible. So I wanted to try and solve the toy linear problem using GaussNewton (basically something where the matrix is actually being formed and inverted) so I can look at things step by step. Unfortunately, when we I set opt = Optimization.GaussNewton(maxIter=60) I get the following error:

'_CustomLinearOperator' object has no attribute 'tocsc'

The SolverWrapper assumes that the matrix being passed to it is sparse. The error occurs when the code in the wrapper tries to set the sparse matrix to a csc_sparse format.

There are a few things going on.

1. The wrapper for direct solvers SolverWrapD assumes the system matrix is sparse.
2. SolverWrapD is currently used for Solver which is wraps scipy.sparse.linalg.spsolve and SolverLU which wraps scipy.sparse.linalg.splu. The splu code is more restrictive and requires a csc or csr sparse matrix. spsolve accepts an ndarray or sparse matrix.
3. SolverLU is used in many places within the code for forward modelling. Solver is only used in Optimization.GaussNewton
4. The argument H that is passed to Solver in generated by evalfunction. Unfortunately (for this case) the function creates a linear operator. It is a relatively easy task to generate a sparse representation of H from the Linear operator.

My solution is to change the existing code

    @Utils.timeIt
    def findSearchDirection(self):
        return Solver(self.H) * (-self.g)

to

    @Utils.timeIt
    def findSearchDirection(self):
         if isinstance(self.H, sp.linalg.LinearOperator):
            self.H = sp.csr_matrix(self.H.dot(np.identity(self.g.size)))
        elif isinstance(self.H, np.ndarray):
            self.H = sp.csr_matrix(self.H)

        return Solver(self.H) * (-self.g)

but maybe the checking isn’t necessary and a simpler fix would be

    @Utils.timeIt
    def findSearchDirection(self):
        self.H = sp.csr_matrix(self.H.dot(np.identity(self.g.size)))
        return Solver(self.H) * (-self.g)

I have made a notebook that shows the error and I wrote code to replicate inv.run and was able to fix the problem.

It is here https://colab.research.google.com/drive/1iGJ7XTTLdxrb0JjJgirnsCTFMoI6hIlp

I have also managed to make this change to my version of the code and if all goes well I am going to try and make a pull request later.

Sean

This is my first attempt at an issue and PR so let me know if I’m doing it right or wrong.

[fourndo]

Awesome @Rockpointgeo . @sgkang and I wrote a simpler implementation with CG solves. Check out Optimization.ProjectedGNCG, I think this should be a lot clearer, closer to what we are used too at GIF

[Rockpointgeo]

Thanks. @fourndo and @lheagy . I agree with worries about dense matrices, I guess I am two decades out of date so I get a bit freaked out by inexact solvers because I never really worked on large problems that required them. As @fourndo observed I am trying to replicate my GIF experience. Perhaps I should move on but don’t like to do so without understanding what is happening.

I added the PR. In all honesty I imagine no one ever came across this because no one uses that method.