Auto scan for hypothesis test inversion (Implemented)

Description

An auto scan feature for hypothesis test inversion can provide faster and more precise upper limits. One good way I’ve found to do this is to use the TOMS748 root finding algorithm.

Describe the solution you’d like

An upperlimit_auto (or some other appropriate name) function that runs an auto scan instead of a linear scan.

Implementation

An implementation is below. I can make a PR including this in pyhf.infer.intevals if this issue has support.

from scipy.optimize import toms748 as _toms748

def upperlimit_auto(data, model, low, high, level=0.05, atol=2e-12, rtol=1e-15):
    """
    Calculate an upper limit interval ``(0, poi_up)`` for a single
    Parameter of Interest (POI) using an automatic scan through
    POI-space, using the TOMS748 algorithm.
    
    ..., mostly copied from upperlimit docstring.
    """

    def f_all(mu):
        return hypotest(mu, data, model, test_stat="qtilde", return_expected_set=True)

    def f(mu, limit=0):
        # Use integers for limit so we don't need a string comparison
        if limit == 0:
            # Obs
            return f_all(mu)[0] - level
        else:
            # Exp
            # (These are in the order -2, -1, 0, 1, 2 sigma)
            return f_all(mu)[1][limit - 1] - level

    tb, _ = get_backend()
    obs = tb.astensor(_toms748(f, low, high, args=(0), k=2, xtol=atol, rtol=rtol))
    exp = [
        tb.astensor(_toms748(f, low, high, args=(i), k=2, xtol=atol, rtol=rtol))
        for i in range(1, 6)
    ]
    return obs, exp

[lukasheinrich]

It would be ince to find a way to be able to inject algorithms that can perform scans without taking them on fully (maybe ismilar to the work on toy calculator API)… scikit optimize has the ask/tell API

https://scikit-optimize.github.io/stable/auto_examples/ask-and-tell.html

probably would fit into pyhf.infer.intervals

[beojan]

I’ve opened a PR with the implementation I have. I included the suggestions @alexander-held made, but otherwise I stuck to the same return format as the linear scan so that anyone using the linear scan can move over easily.