Joint Inversions in SimPEG

Hello, My name is Jae Deok Kim and I am a Master’s student working on Joint Inversions with Dr. Jiajia Sun at the University of Houston.

As far as I know, the joint inversions in SimPEG are inverting for different physical properties (e.g. electrical conductivity and volume) using a single dataset (https://docs.simpeg.xyz/content/examples/11-seis/seis_tomo_joint_with_volume.html). We would like to add the functionalities for joint inversions that combine different physical properties (e.g. gravity and magnetics) from different datasets. This means that we would input multiple datasets of possibly different physical parameters (e.g. gravity and magnetics) and would output two different physical property models (e.g. density and susceptibility) that would be consistent in a predefined way (e.g. petrophysical relationships or structural similarity).

The joint inversion should be done within a single computational algorithm, with a single objective function and where all the models parameters are adjusted concurrently throughout the inversion.

Take for instance, the case of joint inversion of two models. We define our objective function as:

The last term, , is the coupling term that mathematically defines the relationship between different model parameters. The most prominent example would be the cross-gradient.

To minimize this objective function, we can still use the Gauss-Newton method by making some linear algebraic modifications.

Let me first restate the objective function in explicit form:

We can combine this so that, we work with a single vector that represents the stacked models:

Our goal then becomes to find

At each Gauss-Newton step, we solve the system of linear equations given by:

where,

More explicitly, the Hessian can be stated as:

And the gradient can be explicitly stated as:

By constructing the Hessian and gradient as shown, we are able to take advantage of the optimization code that is already in SimPEG to implement our joint inversions.

These are the modifications that I’ve done:

• Coupling: I’ve added a Coupling class that stores the methods related to the specific coupling strategy we use for joint inversions. For now, I’ve only coded the cross-gradient method. The relevant functions are deriv and deriv2 which compute the gradient and Hessian, respectively. For more detailed information on the cross-gradient, you may refer to this paper by Gallardo and Meju (2003) https://doi.org/10.1029/2003GL017370

• Directives: I’ve added a JointInversionDirective class that inherits from InversionDirective, but we define some of the methods differently. Namely, for joint inversions we are now working with multiple data misfits, multiple regularizations, and a coupling term, so I’ve added methods that call and set the values for these objects. I’ve added a SaveOutputEveryIteration_JointInversion directive that will save the outputs of each iteration for joint inversions. I’ve added a UpdatePreconditioner_JointInversion directive that sets a Jacobi preconditioner for the joint inversion. Lastly, I’ve added a Adaptive_Beta_Reweighting directive that will adaptively change the trade-off parameters of the regularizations during the inversion. It makes sure that we stay within a certain range of our target data misfits.

• InvProblem: I’ve created a new JointInvProblem class that inherits from BaseInvProblem. The relevant function is evalFunction, which computes the value of the objective function, as well as the gradient and Hessian (we store the Hessian as a scipy.sparse.linalg.LinearOperator object).

• Optimization: I would like to add another stopping criteria, which I’ve named ratio_x, defined as . This captures information on how much we’ve updated the model, and if the update is negligibly small we decide we’ve converged. It’s somewhat similar to the moving_x stopping criteria, but different in the sense that the ratio_x is like a percentage on the model update at each iteration, where as moving_x is relevant to the initial model. I’ve also added some IterationPrinters that are relevant for the joint inversion, such as printing multiple data misfits, multiple regularizations, the coupling term. Lastly, I’ve modified the ProjectedGNCG findsearchdirection method, where I think it’s more intuitive to use a for loop for i in range(self.maxIterCG) and have a if...then...break statement inside the loop, instead of a while loop. For the convergence criteria of the Conjugate Gradient, I’ve found that using r.dot(r) / self.g.dot(self.g) < self.tolCG works better especially in cases where the residual does not easily fall below magnitudes of 1e-3.

• Inversion: I’ve added a JointInversion class that inherits from BaseInversion that simply sets the IterationPrinters and StoppingCriteria for the joint inversion.

It’s my first time contributing to an open-source project, so any guidance on how to proceed is appreciated (e.g. what tests I need to do?). I’m opening this issue to discuss the possibility of adding these functionalities for joint inversions in SimPEG. I’ve tested these modifications locally and the joint inversion runs successfully, so I’m ready to start a pull request any time, but first I’d like to hear back from the team.

Best regards, Jae

[fourndo]

I am pretty excited about the Coupling part. It has been on my radar for a while to test.

Sounds like you would like to create a new sub-class of Regularization, and you will need a directive to go along with it to update the model. This can be brought in on a single PR with a small example. I can help you with writing a small test that will just check for consistency of results.

[jcapriot]

Most of these changes were brought in in version 0.17.0 which included the start of the Coupling regularizations for joint inversion.