Using an ensemble sampler within a Gibbs step

After talking to @davidwhogg about hierarchical modelling and Gibbs sampling at @astrohackweek, I’ve got some implementation questions. #1. Can ensemble samplers be used in both/all phases of a Gibbs step?

For example, I have ensemble samplers for parameters local to each pixel. Each sampler has N_w walkers. Now how do I pass the last step from the pixel samplers on to the next phase that samples in global parameters? Specifically, what goes in the args to the run_mcmc method? I see two options:

1. Just take the state from a single walker (the first one? or a random one?) and pass that on to the global sampler.
2. If the ensemble samplers at the local and global levels both have N_w walkers then I could ‘align’ the walkers of each. Hence the final state of walker i at the local level would be used by walker i at the global level. I think this is more appealing but is means that I’d need some way of telling the posterior function which walker it belongs to so it can choose the appropriate parameters from the args. @dfm, is there already a way to do this?

Or is it A Bad Idea ™️ to use an ensemble sampler within Gibbs (as I think @dfm alluded in #106, though @davidwhogg seemed optimistic)? #2. How can ensemble samplers be used as a step with a non-ensemble sampler in another step?

If I use an ensemble level for one level of the Gibbs step and a Metropolis-Hastings in another, what parameters do I pass on to MH? Do I always take the parameters at the last step of the i-th walker? Or pick a random walker? (This question mirrors option 1 from above). #3. Is it a special case to have an ensemble sampler step go into a trivial Gaussian Process step?

As @davidwhogg pointed out, in one of my global Gibbs phases I can make a sample from a normal distribution defined by model-observation residuals. In computing these residuals, should I:

1. marginalize over the models of all walkers to make a single new sample
2. just pick a single walker to make a single new sample, or
3. make a new value for each walker (which means that again, the posterior function for the ensemble sampler at the next level would need to know what walker it belongs to).

(This is a question directed mostly to @davidwhogg and @dfm but anyone is welcome to chime in!)

[jonathansick]

Hey @RuthAngus, I ended not using Emcee and instead just implemented my own Metropolis-Hastings in Gibbs (contrary to everyone’s advice to look for another off-the-shelf solution 😉 ).

In my problem with modeling the stellar population in N pixels, with a background parameter that applies to all pixels I do this:

1. For each stellar population parameter of each pixel, I do an MH step to propose a new parameter. So at least I can do my N pixels in parallel, but the MH step is always a single parameter.
2. When the stellar population parameters of each pixel are all updated, I go up a level and estimate a new background. It turns out that I just need a linear estimator for this.
3. Using the new background parameter, I go back to the pixel level and serially do MH steps again through stellar population parameters of each pixel.
4. Re-estimate background level, and repeat.

This ends up being very simple to implement and was really all I needed. I’m not sure if every project could afford to break up their multi-parameter steps into a series of single parameter MH steps, but it worked for me.

[guro86]

Hello, I have a hierarchical problem and am intrigued by using a Gibbs sampler with an inner MH (or emcee) step. I see that you have been discussing this problem since a while back. Does anyone have any example implementation to share? @guillochon, you mentioned that you had success with this, do you still have the code to share?

I really appreciate any help you can provide.