`shgo` error since scipy 1.8.0.dev0+1529.803e52d

shgo raises an error with the following example since scipy 1.8.0.dev0+1529.803e52d.

This error was noticed in the hyperspy test suite (https://github.com/hyperspy/hyperspy-extensions-list/actions/runs/1111267440) and previous scipy development version (1.8.0.dev0+1503.8647ee9) didn’t raise an error. Scipy development version are installed from https://pypi.anaconda.org/scipy-wheels-nightly/simple.

Reproducing code example:

import sys

import numpy as np
import scipy
from scipy.optimize import shgo

print(scipy.__version__, np.__version__, sys.version_info)


def func(x_values, param):
    a, b = param
    return a*x_values + b


def errfunc(param, y):
    return func(x_values, param) - y


def errfunc_sq(param, y, weights=None):
    if weights is None:
        weights = 1.0
    return ((weights * errfunc(param, y)) ** 2).sum()

x_values = np.arange(100)
y_values = func(x_values, (2, 10)) + np.random.random(x_values.size)

results = shgo(errfunc_sq,
               bounds=((-100, 100), (-100, 100)),
               args=(y_values, None)
               )

print(results)

Error message:

1.8.0.dev0+1529.803e52d 1.21.1 sys.version_info(major=3, minor=8, micro=10, releaselevel='final', serial=0)
Traceback (most recent call last):
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo_lib/triangulation.py", line 630, in __getitem__
    return self.cache[x]
KeyError: (0, 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "untitled0.py", line 34, in <module>
    results = shgo(errfunc_sq,
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo.py", line 420, in shgo
    shc.construct_complex()
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo.py", line 733, in construct_complex
    self.iterate()
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo.py", line 876, in iterate
    self.iterate_complex()
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo.py", line 895, in iterate_hypercube
    self.HC = Complex(self.dim, self.func, self.args,
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo_lib/triangulation.py", line 25, in __init__
    self.n_cube(dim, symmetry=symmetry)
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo_lib/triangulation.py", line 76, in n_cube
    self.C0.add_vertex(self.V[origintuple])
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo_lib/triangulation.py", line 634, in __getitem__
    xval = Vertex(x, bounds=self.bounds,
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/_shgo_lib/triangulation.py", line 557, in __init__
    self.f = func(x_a, *func_args)
  File "/opt/miniconda3/envs/test_env/lib/python3.8/site-packages/scipy/optimize/optimize.py", line 464, in function_wrapper
    fx = function(np.copy(x), *(wrapper_args + args))
TypeError: errfunc_sq() takes from 2 to 3 positional arguments but 5 were given

Scipy/Numpy/Python version information:

1.8.0.dev0+1529.803e52d 1.21.1 sys.version_info(major=3, minor=8, micro=10, releaselevel=‘final’, serial=0)

[jjramsey]

I think I’ve found a solution. On line 900 of scipy/optimize/_shgo.py, change

self.HC = Complex(self.dim, self.func, self.args,

to

self.HC = Complex(self.dim, self.func, (),

What’s happening is that since self.func already contains the contents of self.args, when Complex is constructed, it effectively includes self.args twice, so when it calls the objective function, it effectively calls it as func(x, *(args+args)).

Unfortunately, an alternative like setting self.func to _wrap_scalar_function(func, ()) doesn’t work, since there are other places in the code where it’s assumed that self.func wraps the contents of self.args. self.args is also used in other places in the code, so setting it to () probably won’t work either. If the sampling method is changed from the default simplicial, SHGO still appears to work properly, regardless of my change. The change I recommend appears to be the most minimal change needed to get SHGO to work with externally supplied arguments again.

[Stefan-Endres]

Thank you for the report @ericpre. @tupui this is a new bug, but on the old caching please keep the issue open I will fix this in the new caching library.