question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

interp1d issue when trying to fill for values outsite of the data range

See original GitHub issue

The interpolate.interp1d() function with kind='previous' does not work when called for values outside the data range. According to the documentation, if no value is given for fill_value, the outside values should be set to Nan and if a value or tuple of values are given, they should be used to fill the values outside the data range. However, when trying these two techniques, I get an error message. The only way I can make this work on data with values outside the data range is by setting fill_value='extrapolate'.

Reproducing code example:

f = interpolate.interp1d([3,4,5,6], [1,1,1,1], kind='previous', assume_sorted=True, fill_value=(0,5))
#This should fill all values below data range with 0 and above with 5. 
f([0,1,2,3,7])

f = interpolate.interp1d([3,4,5,6], [1,1,1,1], kind='previous', assume_sorted=True)
#This should fill all values outside data range with Nan
f([0,1,2,3,7])

Error message:

  ---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-98-9b5aec5b236e> in <module>
----> 1 f([0,1,2,3,7])

/usr/local/lib/sancare/python/venv/ml-dev/lib/python3.7/site-packages/scipy/interpolate/polyint.py in __call__(self, x)
     77         """
     78         x, x_shape = self._prepare_x(x)
---> 79         y = self._evaluate(x)
     80         return self._finish_y(y, x_shape)
     81 

/usr/local/lib/sancare/python/venv/ml-dev/lib/python3.7/site-packages/scipy/interpolate/interpolate.py in _evaluate(self, x_new)
    661         y_new = self._call(self, x_new)
    662         if not self._extrapolate:
--> 663             below_bounds, above_bounds = self._check_bounds(x_new)
    664             if len(y_new) > 0:
    665                 # Note fill_value must be broadcast up to the proper size

/usr/local/lib/sancare/python/venv/ml-dev/lib/python3.7/site-packages/scipy/interpolate/interpolate.py in _check_bounds(self, x_new)
    690         # !! Could provide more information about which values are out of bounds
    691         if self.bounds_error and below_bounds.any():
--> 692             raise ValueError("A value in x_new is below the interpolation "
    693                              "range.")
    694         if self.bounds_error and above_bounds.any():

ValueError: A value in x_new is below the interpolation range.

Scipy/Numpy/Python version information:

1.4.1 1.18.5 sys.version_info(major=3, minor=7, micro=5, releaselevel='final', serial=0)

Note: I have also reproduced this bug on version 1.5.2 of scipy and it gave the exact same error message

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (4 by maintainers)

github_iconTop GitHub Comments

3reactions
ev-brcommented, Feb 5, 2021

So you’re saying it’s plenty clear already? This trips beginners on a fairly regular basis, so there does seem to be some room for improvement. And this improvement is likely best done by someone who just figured it out: they’re best positioned to judge what would have been most helpful for them.

1reaction
ev-brcommented, Aug 12, 2020

You need to set bounds_error=False explicitly:

In [22]: kinds = ['linear', 'previous', 'next', 3]                                                     

In [23]: for kind in kinds: 
    ...:     fl = interp1d([3,4,5,6], [1,1,1,1], kind=kind, fill_value=(0, 42), bounds_error=False) 
    ...:     print(kind, fl(0), fl(8)) 
    ...:                                                                                               
linear 0.0 42.0
previous 0.0 42.0
next 0.0 42.0
3 0.0 42.0

This can be clarified in the documentation, are interested in sending a pull request?

Read more comments on GitHub >

github_iconTop Results From Across the Web

`ValueError: A value in x_new is above the interpolation range ...
If you are running Scipy v. 0.17.0 or newer, then you can pass fill_value='extrapolate' to spi.interp1d , and it will extrapolate to ...
Read more >
scipy.interpolate.interp1d — SciPy v1.9.3 Manual
If a two-element tuple, then the first element is used as a fill value for ... If “extrapolate”, then points outside the data...
Read more >
How to use the scipy.interpolate.interp1d() method - Educative.io
The fill_value is NaN by default and NaN values are generated every time you try to interpolate y values out of range unless...
Read more >
Introduction to Scipy: Interpolation and Integration
The simplest interpolation routine in scipy.interpolate is interp1d: ... If the interpolating function is called outside the original range, ...
Read more >
scipy.interpolate.interp1d — SciPy v0.18.1 Reference Guide
scipy.interpolate.interp1d¶ · if a ndarray (or float), this value will be used to fill in for requested points outside of the data range....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found