Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

FracdiffStat enormously slow

See original GitHub issue

Hi! I was trying to differentiate an Ethereum price dataset (attached below) but it is taking more than 1h and also uses more than 10 GB of memory. I don’t mind the high memory usage, but it’s taking too much time

from fracdiff import FracdiffStat
import pandas as pd

df = pd.read_csv('ETHUSDT.csv.gz')
FracdiffStat().fit_transform(df.values)

ETHUSDT.csv.gz

Issue Analytics

State:
Created 2 years ago
Comments:9 (9 by maintainers)

Top GitHub Comments

1reaction

cTatucommented, May 1, 2021

Thank you! There is in fact a gpu implementation of KPSS https://github.com/rietmann-nv/cuml/pull/1/files

1reaction

simakicommented, May 1, 2021

Thanks for letting me know!

I think the ADF test (adfuller in statsmodels) is the bottleneck. As you point out, it performs linear regression which needs matrix multiplication and inversion.

Fractional differentiation is the linear order of n_samples (since it is essentially convolution) and order is binary-searched. These operations are optimized and unlikely to be the bottleneck.

Top Results From Across the Web

FracdiffStat - fracdiff documentation - GitHub Pages

A scikit-learn transformer to compute fractional differentiation, where the order is chosen as the minumum order that makes fracdiff stationary. Parameters:.