question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[RFC] Stop multi-thread supports.

See original GitHub issue

Motivation

The current optuna supports multi-thread execution. However, it does not result in any speed gain in general use cases because of python GIL. Rather, the execution time can become longer when we specify n_jobs > 1. The multi-thread support just complicates storage codes and causes hard-to-catch bugs. For example, I suspect the thread-safety relates to #820 Additionally, it forces storage classes to copy data here and there, which increases overhead even at single-thread execution.

Proposal

I’d like to propose removing the whole multi-thread support and leave the n_job argument for future multi-process support. I’d like to propose removing the whole multi-thread support and instead replace it with multi-process execution. It requires multi-process support by InMemoryStorage and some changes in Study logics e.g. study.stop

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:3
  • Comments:13 (7 by maintainers)

github_iconTop GitHub Comments

2reactions
ytsmilingcommented, May 12, 2020

I updated the proposal. Even though the updated proposal is already mentioned in #1170, I think this problem is worth having a dedicated issue to discuss.

0reactions
github-actions[bot]commented, Dec 23, 2021

This issue was closed automatically because it had not seen any recent activity. If you want to discuss it, you can reopen it freely.

Read more comments on GitHub >

github_iconTop Results From Across the Web

RFC: Multi-Thread HDF5 - The HDF Group
This RFC is an attempt to define a strategy for retrofitting thread safety1 on the HDF5 library, that provides immediately useful partial ...
Read more >
RFC 6396 - Multi-Threaded Routing Toolkit (MRT) Routing ...
OSPFv2 Type This type supports the OSPFv2 protocol as defined in RFC 2328 [RFC2328]. It is used to encode the exchange of OSPF...
Read more >
D52193 RFC: [clang] Multithreaded compilation support - LLVM
We've realized that compiling Clang+LLD with Clang generates a faster executable that with MSVC (even latest one). I currently can't see a good ......
Read more >
RFC 47: Per Dataset Caching and GDALRasterBand ...
This is an attempt to make the LRU cache more efficient in multithreaded situations by making it possible to have multiple LRU per...
Read more >
Multi Threading with AutoIt and SAP RFC Library
Home · AutoIt v3 · AutoIt Help and Support · AutoIt General Help and Support; Multi Threading with AutoIt and SAP RFC Library ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found