Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
RuntimeWarning when using pure Python reader with process workers
See original GitHub issueI noticed the following warnings while running benchmarks with pure Pyton reader using process workers (code snippet available in https://github.com/uber/petastorm/issues/584):
/home/dfilipsk/miniconda3/envs/petastorm/lib/python3.6/runpy.py:125: RuntimeWarning: 'petastorm.workers_pool.exec_in_new_process' found in sys.modules after import of package 'petastorm.workers_pool', but prior to execution of 'petastorm.workers_pool.exec_in_new_process'; this may result in unpredictable behaviour
warn(RuntimeWarning(msg))
This warning is repeated as many times as the amount of workers in the pool.
Issue Analytics
- State:
- Created 3 years ago
- Reactions:4
- Comments:6
Top Results From Across the Web
how to get rid of warning `RuntimeWarning: A loop is being ...
The reason you need to do this is that p.terminate() only sends signal.SIGTERM to the process. The process might still do some cleanup...
Read more >Warnings in Python - GeeksforGeeks
Python program terminates immediately if an error occurs. Conversely, a warning is not critical. It shows some message, but the program runs.
Read more >sys — System-specific parameters and functions — Python ...
This module provides access to some variables used or maintained by the interpreter and to functions that interact strongly with the interpreter.
Read more >gevent crash on Python 3.10 (python-x2go fails to build)
This seems to be python or possibly python-gevent crashing when running sphinx-build-3. Running with gdb just results in a corrupted stack ...
Read more >A viable solution for Python concurrency - Hacker News
If you are actually trying to do computationally intensive work, you should really be using multiple processes instead of threads in a ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I can take a look at this later this week. This is something regarding the way we spawn a process-pool worker and the way unpickling works. Feel free to poke that piece of the code if you need it sooner.
petastorm==0.11.4 with the fix is now available on pypi