question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BUG: Series rolling .var with window>14, center=True, and any win_type crashes python with no error message

See original GitHub issue

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

"""
Crash occurs often when:
    window >= 15, 
    center=True,
    win_type='triang' (or any win_type AFAIK),
    usually in the first few trials.

Crash does not result in an error message, python just quits.

Crash does not occur when:
    window <= 14
    win_type is not specified
    center=False
    .var() is changed to .mean()
"""

x = pd.Series(0)

N = 10

for i in range(N):
    print('Starting trial %s out of %s...' % (i+1,N), end='')
    result = x.rolling(window=16, center=True, win_type='triang').var()
    print('success')

Issue Description

This script, if executed, causes python to simply stop (as if calling quit()) ~3/4ths of the time, usually when i > 1.

Crash seems to occur often when:

  • window >= 15,
  • center=True,
  • win_type=‘triang’ (or any win_type AFAIK),

Crash does not seem to occur when:

  • window <= 14
  • win_type is not specified
  • center=False
  • .var() is changed to .mean()

Example of repeatedly running the script:

>>python module1.py
Starting trial 1 out of 10...success
Starting trial 2 out of 10...success
Starting trial 3 out of 10...success
Starting trial 4 out of 10...success

>>python module1.py
Starting trial 1 out of 10...success
Starting trial 2 out of 10...success
Starting trial 3 out of 10...success

>>python module1.py
Starting trial 1 out of 10...success
Starting trial 2 out of 10...success
Starting trial 3 out of 10...success
Starting trial 4 out of 10...success
Starting trial 5 out of 10...success
Starting trial 6 out of 10...success
Starting trial 7 out of 10...success
Starting trial 8 out of 10...success
Starting trial 9 out of 10...success
Starting trial 10 out of 10...success

Expected Behavior

The script should run reliably, or at least deterministically.

An error should appear if there is an issue, but instead python appears to just quit()/crash without any message.

Installed Versions

INSTALLED VERSIONS

commit : 4bfe3d07b4858144c219b9346329027024102ab6 python : 3.10.4.final.0 python-bits : 64 OS : Windows OS-release : 10 Version : 10.0.19044 machine : AMD64 processor : AMD64 Family 25 Model 33 Stepping 0, AuthenticAMD byteorder : little LC_ALL : None LANG : None LOCALE : English_United States.1252

pandas : 1.4.2 numpy : 1.22.3 pytz : 2022.1 dateutil : 2.8.2 pip : 22.0.4 setuptools : 58.1.0 Cython : None pytest : None hypothesis : None sphinx : None blosc : None feather : None xlsxwriter : None lxml.etree : None html5lib : None pymysql : None psycopg2 : None jinja2 : None IPython : None pandas_datareader: None bs4 : None bottleneck : None brotli : None fastparquet : None fsspec : None gcsfs : None markupsafe : None matplotlib : None numba : None numexpr : None odfpy : None openpyxl : None pandas_gbq : None pyarrow : None pyreadstat : None pyxlsb : None s3fs : None scipy : 1.8.0 snappy : None sqlalchemy : None tables : None tabulate : None xarray : None xlrd : None xlwt : None zstandard : None

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:7 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
mroeschkecommented, Apr 13, 2022

Actually since I was running this in ipython, after exiting I get

python(1183,0x118a2adc0) malloc: *** error for object 0x7ff8000000000000: pointer being freed was not allocated
python(1183,0x118a2adc0) malloc: *** set a breakpoint in malloc_error_break to debug
zsh: abort      ipython

Which is probably related to why your program crashes.

0reactions
mroeschkecommented, Apr 13, 2022

I need to make that change in my fork of pandas and build pandas from source to test it?

Correct, https://pandas.pydata.org/pandas-docs/stable/development/contributing_environment.html#creating-a-python-environment

Read more comments on GitHub >

github_iconTop Results From Across the Web

BUG: Offset-based rolling window, with closed='left' and max ...
Problem description. With this rolling and aggregation function, python just crashes. It does too with .min() or .agg(np.max) as aggregation ...
Read more >
pandas.Series.rolling — pandas 1.5.2 documentation
For a DataFrame, a column label or Index level on which to calculate the rolling window, rather than the DataFrame's index.
Read more >
Use center in pandas rolling when using a time-series
This will center your rolling sum in the 7-day window (by shifting ... 'D')) series.rolling(7, min_periods=1, center=True).sum().head(10).
Read more >
Issue 10365: IDLE Crashes on File Open Dialog when code ...
Open IDLE (Python 3.1.2) 2. Open a .py file 3. With the code window (not the shell window) in focus, Ctrl + O...
Read more >
Python | Pandas dataframe.rolling() - GeeksforGeeks
The concept of rolling window calculation is most primarily used in signal processing and time-series data. In very simple words we take a ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found