Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Investigate `WARNING:tensorflow:multiprocessing can interact badly with TensorFlow`
See original GitHub issuehinsage/movielens-recommender.ipynb has the following warning:
WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing
nondeterministic deadlocks. For high performance data pipelines tf.data is recommended
It seems like this warning will pop up whenever someones uses use_multiprocessing=True with any of our keras generators. Replacing our generators with tf.data will fix this warning however this will take a significant amount of work. This issue is to explore whether its worth replacing our generators with tf.data objects.
Issue Analytics
- State:
- Created 4 years ago
- Comments:6 (3 by maintainers)
Top Results From Across the Web
Using tf.keras.utils.Sequence with model.fit_generator with ...
WARNING :tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data ...
Read more >Speed Up your Keras Sequence Pipeline | by Mudit Bachhawat
WARNING:tensorflow:multiprocessing can interact badly with TensorFlow, causing nondeterministic deadlocks. For high performance data pipelines ...
Read more >Understanding Performance Problems in Deep Learning ...
“WARNING: tensorflow: multiprocessing can interact badly with Ten-. sorFlow, causing nondeterministic deadlocks. For high performance.
Read more >NVIDIA Deep Learning TensorRT Documentation
This NVIDIA TensorRT Developer Guide demonstrates how to use the C++ and Python APIs for implementing the most common deep learning layers.
Read more >How to Grid Search Hyperparameters for Deep Learning ...
Keras models can be used in scikit-learn by wrapping them with the KerasClassifier or ... from tensorflow.keras.models import Sequential.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Potentially we can resolve by simply not using multiprocessing
I use keras.fit_generator and multiprocessing = True.I have the same error too