RFE: n_jobs should accommodate loadavg
See original GitHub issueOften I want to start many independent processes on a shared server, and I cannot know in advance how many CPUs will be available to Parallel
.
I want to be able to tell it to start new process whenever, say, getloadavg1
< 1.5*cpu_count
(as opposed to a fixed n_jobs
)
Issue Analytics
- State:
- Created 4 years ago
- Comments:6 (3 by maintainers)
Top Results From Across the Web
5.2.16. /proc/loadavg Red Hat Enterprise Linux 4
/proc/loadavg. This file provides a look at the load average in regard to both the CPU and IO over time, as well as...
Read more >Understanding Linux CPU Load - when should you be worried?
First, a little background on what the load average values mean. We'll start out with the simplest case: a machine with one single-core ......
Read more >What do the numbers in /proc/loadavg mean on Linux?
The first of these is the number of currently executing kernel scheduling entities (processes, threads); this will be less than or equal to...
Read more >UNIX Load Average Part 1: How It Works - Fortra
In Solaris 2.0 and 2.2 the load average did not include the running jobs but this bug was fixed in Solaris 2.3. So,...
Read more >What is Load Average in Linux? - DigitalOcean
Similarly, it would represent 25% CPU usage for a quad-core processor. Load Average in Linux takes into account the waiting threads and tasks ......
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
https://github.com/joblib/loky/issues/233
… including the server I share with myself and run many parallel jobs