Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

.rolling().std() only returns NaN in Python3.7

See original GitHub issue

import pandas as pd
d = {"col": [1, 23, 231, 231, 4, 353, 62, 3, 56, 43, 354, 43, 231, 21, 7]}
df = pd.DataFrame(data=d)
std = df["col"].std()
df["mean5"] = df["col"].rolling(5).mean()
df["std5"] = df["col"].rolling(5).std()

print(std)
print(df[["mean5", "std5"]])

# OUTPUT
130.20855066648528
    mean5  std5 
0     NaN   NaN
1     NaN   NaN
2     NaN   NaN
3     NaN   NaN
4    98.0   NaN
5   168.4   NaN
6   176.2   NaN
7   130.6   NaN
8    95.6   NaN
9   103.4   NaN
10  103.6   NaN
11   99.8   NaN
12  145.4   NaN
13  138.4   NaN
14  131.2   NaN

Problem description

.std() and .rolling().mean() work as intended, but .rolling().std() only returns NaN I just upgraded from Python 3.6.5 where the same code did work perfectly. I am now on Python 3.7, pandas 0.23.2

Expected Output

130.20855066648528
    mean5        std5
0     NaN         NaN
1     NaN         NaN
2     NaN         NaN
3     NaN         NaN
4    98.0  108.855868
5   168.4  134.226078
6   176.2  126.458531
7   130.6  138.965607
8    95.6  131.085621
9   103.4  126.482568
10  103.6  126.877264
11   99.8  128.342355
12  145.4  126.337010
13  138.4  131.941805
14  131.2  137.803338

Output of `pd.show_versions()`

INSTALLED VERSIONS

commit: None python: 3.7.0.final.0 python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 94 Stepping 3, GenuineIntel byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: None.None

pandas: 0.23.2 pytest: None pip: 10.0.1 setuptools: 39.0.1 Cython: None numpy: 1.14.5 scipy: None pyarrow: None xarray: None IPython: None sphinx: None patsy: None dateutil: 2.7.3 pytz: 2018.5 blosc: None bottleneck: None tables: None numexpr: None feather: None matplotlib: None openpyxl: None xlrd: None xlwt: None xlsxwriter: None lxml: None bs4: 4.6.0 html5lib: None sqlalchemy: None pymysql: None psycopg2: None jinja2: None s3fs: None fastparquet: None pandas_gbq: None pandas_datareader: None None

Issue Analytics

State:
Created 5 years ago
Comments:19 (10 by maintainers)

Top GitHub Comments

1reaction

tweakimpcommented, Jul 8, 2018

Workaround:

.apply(lambda x: pd.np.std(x))

import pandas as pd

d = {"col": [1, 23, 231, 231, 4, 353, 62, 3, 56, 43, 354, 43, 231, 21, 7]}
df = pd.DataFrame(data=d)
df["std5"] = df["col"].rolling(5).apply(lambda x: pd.np.std(x))
print(df["std5"])
# OUTPUT (as expected)
0            NaN
1            NaN
2            NaN
3            NaN
4     108.855868
5     134.226078
6     126.458531
7     138.965607
8     131.085621
9     126.482568
10    126.877264
11    128.342355
12    126.337010
13    131.941805
14    137.803338

0reactions

sherwanycommented, Aug 7, 2020

Hi, Wondering if someone can please help me. I am getting the same issue - using pandas 1.0.3 as well as 0.23.2 - I have tried with Python 3.7 and 3.6.5. The workaround mentioned above i.e. .apply(lambda x: pd.np.std(x)) doesn’t fix the issue for me.

SampleData

Is there any other workaround I could use?

Top Results From Across the Web

Pandas rolling gives NaN - python - Stack Overflow

The first thing to notice is that by default rolling looks for n-1 prior rows of data to aggregate, where n is the...

pandas.DataFrame.rolling — pandas 1.5.2 documentation

This argument is only implemented when specifying engine='numba' in the method call. Returns. Window subclass if a win_type is passed: Rolling subclass ...

How to calculate MOVING AVERAGE in a Pandas DataFrame?

In Python, we can calculate the moving average using .rolling() method. This method provides rolling windows over the data, and we can use ......

Rolling Regression - Statsmodels

Rolling OLS applies OLS across a fixed windows of observations and then rolls (moves or ... pd.plotting.register_matplotlib_converters() %matplotlib inline.

The pandas DataFrame Object - Cheat Sheet

Note: the parse() method takes many arguments like ... df['new_col'] = np.repeat(np.nan,len(df)) ... Trap: When adding a new column, only items from the....