Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Interpolate NaT

See original GitHub issue

Hello,

interpolate doesn’t work with NaT see http://stackoverflow.com/questions/33921795/fill-timestamp-nat-with-a-linear-interpolation/33922824#33922824

Here is a trivial example to show the situation:

s = pd.Series(pd.date_range('2015-01-01' , '2015-01-30'), name='t')

s[3], s[4], s[5] = pd.NaT, pd.NaT, pd.NaT
s[13], s[14], s[15] = pd.NaT, pd.NaT, pd.NaT
print(s)

0    2015-01-01
1    2015-01-02
2    2015-01-03
3           NaT
4           NaT
5           NaT
6    2015-01-07
7    2015-01-08
8    2015-01-09
9    2015-01-10
10   2015-01-11
11   2015-01-12
12   2015-01-13
13          NaT
14          NaT
15          NaT
16   2015-01-17
17   2015-01-18
18   2015-01-19
19   2015-01-20
20   2015-01-21
21   2015-01-22
22   2015-01-23
23   2015-01-24
24   2015-01-25
25   2015-01-26
26   2015-01-27
27   2015-01-28
28   2015-01-29
29   2015-01-30
Name: t, dtype: datetime64[ns]

print(s.interpolate())
0    2015-01-01
1    2015-01-02
2    2015-01-03
3           NaT
4           NaT
5           NaT
6    2015-01-07
7    2015-01-08
8    2015-01-09
9    2015-01-10
10   2015-01-11
11   2015-01-12
12   2015-01-13
13          NaT
14          NaT
15          NaT
16   2015-01-17
17   2015-01-18
18   2015-01-19
19   2015-01-20
20   2015-01-21
21   2015-01-22
22   2015-01-23
23   2015-01-24
24   2015-01-25
25   2015-01-26
26   2015-01-27
27   2015-01-28
28   2015-01-29
29   2015-01-30
Name: t, dtype: datetime64[ns]

assert s.interpolate().isnull().sum() == 0
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-150-8a59e397a174> in <module>()
----> 1 assert s.interpolate().isnull().sum() == 0

AssertionError:

Kind regards

Issue Analytics

State:
Created 8 years ago
Comments:7 (6 by maintainers)

Top GitHub Comments

6reactions

jrebackcommented, Sep 28, 2017

This is not very hard to actually do directly (and what .interpolate() should basically do, PRs welcome)

In [12]: s2 = s.astype('i8').astype('f8')

In [13]: s2[s.isnull()] = np.nan

In [14]: pd.to_datetime(s2.interpolate())
Out[14]: 
0    2015-01-01
1    2015-01-02
2    2015-01-03
3    2015-01-04
4    2015-01-05
5    2015-01-06
6    2015-01-07
7    2015-01-08
8    2015-01-09
9    2015-01-10
10   2015-01-11
11   2015-01-12
12   2015-01-13
13   2015-01-14
14   2015-01-15
15   2015-01-16
16   2015-01-17
17   2015-01-18
18   2015-01-19
19   2015-01-20
20   2015-01-21
21   2015-01-22
22   2015-01-23
23   2015-01-24
24   2015-01-25
25   2015-01-26
26   2015-01-27
27   2015-01-28
28   2015-01-29
29   2015-01-30
Name: t, dtype: datetime64[ns]

0reactions

mroeschkecommented, Mar 31, 2020

Looks to be a duplicate of https://github.com/pandas-dev/pandas/issues/11312

Top Results From Across the Web

Fill Timestamp NaT with a linear interpolation - Stack Overflow

You are right about interpolate method currently not working with Timestamp . One solution is to convert it to float, interpolate it and ......

Working with missing data — pandas 1.5.2 documentation

For datetime64[ns] types, NaT represents missing values. ... Both Series and DataFrame objects have interpolate() that, by default, performs linear ...

How to interpolate under specific condition? - MATLAB Answers

If the gap is larger (>4days), I want NaN values to be preserved. Column A contains the dates I have temperature observations (NaT...

A two-dimensional interpolation function for irregularly-spaced ...

ACM '68: Proceedings of the 1968 23rd ACM national conferenceJanuary 1968 Pages ... areal data there arises a need for interpolating from irregularly-spaced ......

The National Meteorological Center's spectral statistical ...

This analysis system is called the Spectral Statistical Interpolation (SS1) Analysis system because the spectral coefficients used in the NMC spectral model are ......