Add unconstrained/transformed variables to InferenceData

@ahartikainen suggested adding unconstrained variables to the schema somewhere. It seems like they could go

1. Make new unconstrained_posterior group Good:
   • Explicit is better than implicit: it is clear what this group is for Bad:
   • Should there also be unconstrained_prior?
2. Add them to the existing posterior group Good:
   • Can reuse existing coords/dims
   • Can add to prior in the same way Bad:
   • Would need a naming convention, which we’ve managed to avoid so far
   • PyMC3 currently does this, and it is kind of a pain to have a flag everywhere to include or exclude these variables.
3. Don’t include them at all Good:
   • No work 😃 Bad:
   • Bounds and domain information are not captured anywhere currently

[aseyboldt]

I would also like to see a way to get the transformed variables in the trace. The fact that this isn’t possible right now is pretty much the only reason I still use the original trace object of pymc3 quite a bit. I think I might even prefer if one of option 1 or 2 was the default behavior. The most common use cases I can see would be

• When diagnosing convergence issues the untransformed space often doesn’t tell you much. If the transformation is just a log transform, that is easy to fix, but say for a Dirichlet this isn’t the case anymore. I think we should always think in the transformed space when making scatter plots of the posteriors, looking for funnels, correlations etc.
• Anything that uses the logp has to also take into account which space we are talking about. The logp that is stored in the trace is with respect to the unconstrained space, so it doesn’t even make much sense when you don’t know what this is.
• If we interact with the model a bit more, because for example we want to also look at the gradients of the logp (this can tell us something about correlations), or if we want to compute sensitivities with respect to some other variable, we need to interact with the aesara graph of the logp function. This is defined in terms of the transformed variables though. Getting at the gradients is easy with the original trace:

func = model.logp_dlogp_function()
func.set_extra_values({})
for point in trace.points():
    logp, grad = func(func.dict_to_array(point))

This is a lot more work if if we do not have the pymc3 trace, because we first have to manually map the values to the transformed space.

[ColCarroll]

hey dan! I think I’m generally on team option 3 still, but also on team “lets help out science”. So I’m totally on board with you adding a little backdoor option to allow this on pymc3, and if you do, maybe I can open a second issue requesting help to add it to various other backends (or even rethinking the design)?