UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak. "timeout or by a memory leak."

Description

Fitting a GridSearchCV model with n_jobs = -1 gives the UserWarning mentioned below, while for n_jobs = 1 it runs without any warnings

Steps/Code to Reproduce

pipeline = Pipeline([('vect', CountVectorizer(tokenizer=tokenise)),
    ('tfidf', TfidfTransformer()),
    ('clf', MultiOutputClassifier(RandomForestClassifier(n_estimators=20, n_jobs=-1)))])

parameters = {'clf__estimator__max_depth': [10, 15],
              'clf__estimator__n_estimators': [100, 300]}
              
model = GridSearchCV(pipeline, parameters, make_scorer(fbeta_score, beta=2, average='macro'), -1, cv=2, verbose=3)

model.fit(X_train, y_train)

Expected Results

No error is thrown

Actual Results

/usr/local/lib/python3.6/dist-packages/joblib/externals/loky/process_executor.py:706: UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak. "timeout or by a memory leak.", UserWarning

Versions

System: python: 3.6.8 (default, Jan 14 2019, 11:02:34) [GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] executable: /usr/bin/python3 machine: Linux-4.14.79±x86_64-with-Ubuntu-18.04-bionic

Python deps: pip: 19.2.1 setuptools: 41.0.1 sklearn: 0.21.3 numpy: 1.16.4 scipy: 1.3.0 Cython: 0.29.13 pandas: 0.24.2

[anjani-dhrangadhariya]

n_jobs=-1 causes the model to use as many available CPU resources as possible. It is better to keep this variable to -10 or even lesser in case you are sharing a server with your lab members. More information here.

[yiorgos1973]

I have this error frequently when using all processors or almost all processors. Anaconda IDEs have a little matter in memory allocation and I suspect that this has also to do with that. Of course it is not an error and the results are not alterd.