Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Reintroduce the feature of n_jobs (multi-thread)

See original GitHub issue

Motivation

The feature of n_jobs (multi-thread) was deprecated in this PR because it is impractical in most use cases and can be reproduced by writing a few lines of code if users want to reproduce the same thing. However, some people on Twitter and 3 people on the PR wanted to reintroduce the feature. Based on the above, it is necessary to consider whether to reintroduce n_jobs (multi-thread) or not.

Description

We would like to make a consensus whether to reintroduce n_jobs (multi-thread) or not. If reintroducing it, we will restore the n_jobs (multi-thread) argument. If not, we will consider appropriate alternatives and develop a plan for them. Also, we will decide that Optuna’s data structures may continue to support multi-threading.

Consensus for the solution (Added on Dec 17, 2011)

From the following reasons, we decided that we will continue to support the feature of n_jobs.

We observed that more people wish to reintroduce this feature than people who claimed for misusing the feature.
Also, this feature can be useful for tasks not blocked by GIL.

TODOs

Make a consensus for the solution in this issue
If reintroducing it,
- Remove deprecate decorator
- Provide enough information to avoid misuse of the feature e.g.
  - API reference
  - FAQ
  - etc
~If not,~
- ~[ ] Create a plan to provide alternative solutions and documentation~
- ~[ ] Make a consensus for Optuna’s data structures~

Reference

https://github.com/optuna/optuna/issues/2950

Issue Analytics

State:
Created 2 years ago
Comments:6 (5 by maintainers)

Top GitHub Comments

1reaction

keisuke-umezawacommented, Jan 11, 2022

@keisuke-umezawa What other documentation needs to be enhanced besides the API reference and FAQ? (The etc part in the issue description.)

Maybe, nothitng.

1reaction

tsugacommented, Sep 21, 2021

I’m wondering if process-based (by multiprocessing module) would be an option as well. This is the code that is for Scikit-Learn integration, but it would great if n_jobs is supported again globally.

https://github.com/tsuga/optuna/blob/55d809f16baf824a758a7e242d7e78484d3541e3/optuna/integration/sklearn.py#L890-L906

Top Results From Across the Web

Multithreading and concurrency fundamentals - Educative.io

Multithreading is a technique that allows for concurrent (simultaneous) execution of two or more parts of a program for maximum utilization of a ......

Unity Job System: Safe and Easy Multithreading in Unity

Sign up for the Level 2 Game Dev Newsletter: http://eepurl.com/gGb8ePThe Unity Job System makes it safe and easy to write multithreaded code ...

Node.js multithreading: Worker threads and why they matter

Learn about worker_threads, the Web Workers API, and find some inspiration for how to use web workers to handle complex tasks.

4. Eight Simple Rules for Designing Multithreaded Applications

Temporary work variables are rarely shared between threads, and should be declared or allocated locally to each thread. Variables that hold partial results...

Using the Multi-thread Execution feature to run Jobs in parallel

Based on the previous use case Using the output stream feature, this use case give an example of how to use the Multi-thread...