Minimization problem with pyhf.infer.mle.fit(data, model, return_uncertainties=True)

Description

Dear pyhf developpers,

I have a rather complex workspace and I wanted to make a pull plot based on the tutorial. I get a crash in the minimization step.

$ python test.py

open filename=  ex2.txt
Calculate the observed and expected limit: 
/home/tcarli/pyhftest/pyhf-env/lib/python3.8/site-packages/pyhf/tensor/numpy_backend.py:353: RuntimeWarning: invalid value encountered in log
  return n * np.log(lam) - lam - gammaln(n + 1.0)
      Observed CLs: 0.0009
Expected CLs(-2 σ): 0.0000
Expected CLs(-1 σ): 0.0004
Expected CLs( 0 σ): 0.0043
Expected CLs( 1 σ): 0.0380
Expected CLs( 2 σ): 0.2017
     corr: None
      fun: 827.5196362158953
 hess_inv: None
  message: 'Optimization failed. Call limit was reached.'
   minuit: <FMin algorithm='Migrad' edm=0.00018604103640955825 edm_goal=2e-06 errordef=1.0 fval=827.5196362158953 has_accurate_covar=False has_covariance=True has_made_posdef_covar=False has_parameters_at_limit=True has_posdef_covar=True has_reached_call_limit=True has_valid_parameters=False hesse_failed=False is_above_max_edm=False is_valid=False nfcn=100392 ngrad=0 reduced_chi2=nan>
...
554527922071 x335=7.865288459575842e-05 x336=0.0003142268985029573 x337=0.00010332798675705222 x338=0.009556615091181594>
Traceback (most recent call last):
  File "/home/tcarli/pyhftest/pyhf-env/lib/python3.8/site-packages/pyhf/optimize/mixins.py", line 49, in _internal_minimize
    assert result.success
AssertionError
Traceback (most recent call last):
  File "/home/tcarli/pyhftest/pyhf-env/lib/python3.8/site-packages/pyhf/optimize/mixins.py", line 49, in _internal_minimize
    assert result.success
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 55, in <module>
    result = pyhf.infer.mle.fit(data, model, return_uncertainties=True)
  File "/home/tcarli/pyhftest/pyhf-env/lib/python3.8/site-packages/pyhf/infer/mle.py", line 131, in fit
    return opt.minimize(
  File "/home/tcarli/pyhftest/pyhf-env/lib/python3.8/site-packages/pyhf/optimize/mixins.py", line 160, in minimize
    result = self._internal_minimize(**minimizer_kwargs, options=kwargs)
  File "/home/tcarli/pyhftest/pyhf-env/lib/python3.8/site-packages/pyhf/optimize/mixins.py", line 52, in _internal_minimize
    raise exceptions.FailedMinimization(result)

This might not be a bug, but rather a problem in the set-up. May be there is some better setting for the minimize ?

Steps to Reproduce

You just need to do $ python test.py and download my workspace ex2.txt

I use version: pyhf, version 0.6.2

Regards, Tancredi

[alexander-held]

Hi @tancredicarli, the default number of allowed iterations in the minimization is 100k. If that is reached, the above error occurs. You can pass a maxiter argument to pyhf.infer.mle.fit to increase the allowed number of iterations:

pyhf.infer.mle.fit(data, model, maxiter=1000000)

When setting the minimizer, you can also set the verbosity to see more MINUIT output, including updates for the number of calls used:

pyhf.set_backend("numpy", pyhf.optimize.minuit_optimizer(verbose=2))

[tancredicarli]

Hello @alexander-held

thank you very for your help. There was indeed some misunderstanding from my side about the meaning of the modifiers. I corrected this exupdatederrorconvention.txt However, I still have a problem (output testoutputexupdatederrorconvention.txt). May be it related to my extreme set-up.I will investigate a bit more where the Chi=nan might come from.

For your second suggestion: Could you help me to set-up the fit correctly fixing mu=0 ? I followed your suggestion in exupdatedpoi.txt

With this work-space I get:

Calculate the observed and expected limit: 
Traceback (most recent call last):
  `File "test.py", line 29, in <module>
    CLs_obs, CLs_exp = pyhf.infer.hypotest(
  File "/home/tcarli/pyhftest/pyhf-env/lib/python3.8/site-packages/pyhf/infer/__init__.py", line 147, in hypotest
    _check_hypotest_prerequisites(pdf, data, init_pars, par_bounds, fixed_params)
  File "/home/tcarli/pyhftest/pyhf-env/lib/python3.8/site-packages/pyhf/infer/__init__.py", line 15, in _check_hypotest_prerequisites
    raise exceptions.InvalidModel(
pyhf.exceptions.InvalidModel: POI at index [0] is set as fixed, which makes inference impossible. Please unfix the POI to continue.

May be have now some inconsistency with the function call ?

The output of python test.py with updated workspace exupdatederrorconvention.txt: testoutputexupdatederrorconvention.txt The updated work-spaces:

1. Updated errors: Modifiers are now varied cross-sections not absolute errors: exupdatederrorconvention.txt

2. Update the errors and treatment of the parameter of interest according to Alex suggestions: exupdatedpoi.txt

Thank you for all the help. Tancredi