Unable to run cross-validation in parallel mode "processes"
See original GitHub issueHello, I’m using Prophet v1.0 with Anaconda3 2020.11 on Windows 10 64-bit. I’m trying to run cross-validation in parallel mode “processes” using the example provided in the documentation, but I always get this error message (the error.log is very long so I attached it instead of pasting it here). The code I used:
import pandas as pd
import itertools
import numpy as np
from prophet import Prophet
from prophet.diagnostics import cross_validation
from prophet.diagnostics import performance_metrics
df = pd.read_csv("example_wp_log_peyton_manning.csv")
param_grid = {
"changepoint_prior_scale": [0.001, 0.01, 0.1, 0.5],
"seasonality_prior_scale": [0.01, 0.1, 1.0, 10.0],
}
# Generate all combinations of parameters
all_params = [dict(zip(param_grid.keys(), v)) for v in itertools.product(*param_grid.values())]
rmses = [] # Store the RMSEs for each params here
# Use cross validation to evaluate all parameters
for params in all_params:
m = Prophet(**params).fit(df) # Fit model with given params
df_cv = cross_validation(m, horizon="30 days", parallel="processes")
df_p = performance_metrics(df_cv, rolling_window=1)
rmses.append(df_p["rmse"].values[0])
# Find the best parameters
tuning_results = pd.DataFrame(all_params)
tuning_results["rmse"] = rmses
print(tuning_results)
If I run the code on Google Colab then everything is fine.
So can anyone help please? Thank you.
Issue Analytics
- State:
- Created 2 years ago
- Reactions:1
- Comments:11 (3 by maintainers)
Top Results From Across the Web
Diagnostics | Prophet - Meta Open Source
Prophet is a forecasting procedure implemented in R and Python. ... Cross-validation can also be run in parallel mode in Python, by setting...
Read more >sklearn.cross_validation.cross_val_score multiple cpu?
I am trying to get a score for a model through cross validation with sklearn.cross_validation.cross_val_score. According to ...
Read more >4 Cross Validation Methods | Introduction to Applied Machine ...
We use cross validation for two goals: To select among model configurations; To evaluate the performance of our models in new data.
Read more >Training modes and algorithm support - Amazon SageMaker
To find the best combination for your dataset, ensemble mode runs 10 trials with different model and meta parameter settings. Then Autopilot combines...
Read more >k-fold cross-validation explained in plain English
The cv hyperparameter represents the number of folds (here it is 5). By providing appropriate values to n_jobs, we can do parallel computations....
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Thanks for your suggestion. Unfortunately the
fork
value is available on Unix only and it’s the default on Unix. On Windows the only available value isspawn
. This information is available in the Python’s official document too.Trying to set the argument to
fork
in Windows will result in this error message:The issue lies in the line
pool = concurrent.futures.ProcessPoolExecutor()
in the filediagnostics.py
. As shared in an answer to a question on StackOverflow on parallelism on Windows:According to #1434, it seems running cross-validation with
parallel
set tothreads
is much slower than setting it toprocesses
. I’m unable to buildprophet
on Windows so my another workaround is to use the WSL. In my case there is a huge difference in term of execution time:parallel
set toprocesses
in WSL (Debian, without Anaconda): 0:02:44.19 (100% CPU usage)parallel
set tothreads
in Windows (with Anaconda): 0:08:40.13 (only around 50% CPU usage)Hope that in the future there will be someone who can help to debug this issue.