question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

RFE: n_jobs should accommodate loadavg

See original GitHub issue

Often I want to start many independent processes on a shared server, and I cannot know in advance how many CPUs will be available to Parallel. I want to be able to tell it to start new process whenever, say, getloadavg1 < 1.5*cpu_count (as opposed to a fixed n_jobs)

Issue Analytics

  • State:open
  • Created 4 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
sam-scommented, Feb 13, 2020

Could you please open an issue for https://github.com/joblib/loky ? This is the default process-based backend for joblib.

https://github.com/joblib/loky/issues/233

0reactions
sam-scommented, Feb 13, 2020

I agree it would be a great feature for shared servers.

… including the server I share with myself and run many parallel jobs

Read more comments on GitHub >

github_iconTop Results From Across the Web

5.2.16. /proc/loadavg Red Hat Enterprise Linux 4
/proc/loadavg. This file provides a look at the load average in regard to both the CPU and IO over time, as well as...
Read more >
Understanding Linux CPU Load - when should you be worried?
First, a little background on what the load average values mean. We'll start out with the simplest case: a machine with one single-core ......
Read more >
What do the numbers in /proc/loadavg mean on Linux?
The first of these is the number of currently executing kernel scheduling entities (processes, threads); this will be less than or equal to...
Read more >
UNIX Load Average Part 1: How It Works - Fortra
In Solaris 2.0 and 2.2 the load average did not include the running jobs but this bug was fixed in Solaris 2.3. So,...
Read more >
What is Load Average in Linux? - DigitalOcean
Similarly, it would represent 25% CPU usage for a quad-core processor. Load Average in Linux takes into account the waiting threads and tasks ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found