Staterror issues with partially fixed bins due to to zero yields

Summary

The changes to staterror handling after 0.6.3 can result in some crashes due to unsupported treatment of partially fixed parameters (fixed modifiers in some bins but not in others). This results in https://github.com/scikit-hep/pyhf/blob/6f8d87e65e2b9b6033974ef18cbb4d9bf62f3dd8/src/pyhf/parameters/paramsets.py#L44-L46 getting raised.

cc @ekourlit

OS / Environment

n/a

Steps to Reproduce

import pyhf

# zero nominal yield, non-zero staterror
spec = {
    "channels": [
        {
            "name": "SR",
            "samples": [
                {
                    "data": [0.0, 1.0],
                    "modifiers": [
                        {
                            "data": [0.1, 0.1],
                            "name": "staterror_SR",
                            "type": "staterror"
                        }
                    ],
                    "name": "sample_1"
                }
            ]
        }
    ],
    "measurements": [{"config": {"parameters": [], "poi": ""}, "name": "measurement"}],
    "observations": [{"data": [0.0, 1.0], "name": "SR"}],
    "version": "1.0.0"
}
model = pyhf.Workspace(spec).model()
for parameter in model.config.par_order:
    try:
        print(parameter, model.config.param_set(parameter).suggested_fixed_as_bool)
    except RuntimeError as e:
        print(e)


# non-zero nominal yield, zero staterror
spec = {
    "channels": [
        {
            "name": "SR",
            "samples": [
                {
                    "data": [0.1, 1.0],
                    "modifiers": [
                        {
                            "data": [0.0, 0.1],
                            "name": "staterror_SR",
                            "type": "staterror"
                        }
                    ],
                    "name": "sample_1"
                }
            ]
        }
    ],
    "measurements": [{"config": {"parameters": [], "poi": ""}, "name": "measurement"}],
    "observations": [{"data": [0.0, 1.0], "name": "SR"}],
    "version": "1.0.0"
}
model = pyhf.Workspace(spec).model()
for parameter in model.config.par_order:
    try:
        print(parameter, model.config.param_set(parameter).suggested_fixed_as_bool)
    except RuntimeError as e:
        print(e)


# zero nominal yield, zero staterror
spec = {
    "channels": [
        {
            "name": "SR",
            "samples": [
                {
                    "data": [0.0, 1.0],
                    "modifiers": [
                        {
                            "data": [0.0, 0.1],
                            "name": "staterror_SR",
                            "type": "staterror"
                        }
                    ],
                    "name": "sample_1"
                }
            ]
        }
    ],
    "measurements": [{"config": {"parameters": [], "poi": ""}, "name": "measurement"}],
    "observations": [{"data": [0.0, 1.0], "name": "SR"}],
    "version": "1.0.0"
}
model = pyhf.Workspace(spec).model()
for parameter in model.config.par_order:
    try:
        print(parameter, model.config.param_set(parameter).suggested_fixed_as_bool)
    except RuntimeError as e:
        print(e)


# example that works: non-zero data and zero staterror for one sample,
# zero data and non-zero staterror for another sample
spec = {
    "channels": [
        {
            "name": "SR",
            "samples": [
                {
                    "data": [0.1, 1.0],
                    "modifiers": [
                        {
                            "data": [0.0, 0.1],
                            "name": "staterror_SR",
                            "type": "staterror"
                        }
                    ],
                    "name": "sample_1"
                },
                {
                    "data": [0.0, 1.0],
                    "modifiers": [
                        {
                            "data": [0.1, 0.1],
                            "name": "staterror_SR",
                            "type": "staterror"
                        }
                    ],
                    "name": "sample_2"
                }
            ]
        }
    ],
    "measurements": [{"config": {"parameters": [], "poi": ""}, "name": "measurement"}],
    "observations": [{"data": [0.0, 1.0], "name": "SR"}],
    "version": "1.0.0"
}
model = pyhf.Workspace(spec).model()
for parameter in model.config.par_order:
    try:
        print(parameter, model.config.param_set(parameter).suggested_fixed_as_bool)
    except RuntimeError as e:
        print(e)

File Upload (optional)

No response

Expected Results

The script above should run without running into any errors.

Actual Results

[True, False] is neither all-True nor all-False, so not compressible
[True, False] is neither all-True nor all-False, so not compressible
[True, False] is neither all-True nor all-False, so not compressible
staterror_SR False

pyhf Version

pyhf, version 0.7.0rc2.dev18

Code of Conduct

• I agree to follow the Code of Conduct

[matthewfeickert]

I was wondering whether this is related to that issue.

I’m pretty sure it is different. As we clearly have problems with staterror I’m more just trying to loosely connect them together for easier reference. I’ll just make an Issue for that now though and do it properly.

[kratsg]

Closing this as “won’t fix”.