question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Storage slowing down execution

See original GitHub issue

Hi,

I have some troubles to make Optuna runs in parallel, and even in sequential but with a local storage. My test code is really simple:

import optuna

def objectif(trial):
    x = trial.suggest_float("x", -10., 10.)
    y = trial.suggest_float("y", -10., 10.)

    return (x - 5)**2 + (y + 4)**2

study = optuna.create_study(direction = "minimize", study_name = "TestOptuna",\
                            load_if_exists = True)
study.optimize(objectif, n_trials = 1000)

Executing the previous code takes : 8,83s. Now if I add some local storage with :

study = optuna.create_study(direction = "minimize", study_name = "TestOptuna",\
                            load_if_exists = True, storage = "mysql://root:#password@localhost/TestOptuna")

The execution takes : 2min6,6s. Same problem with sqlite.

Any idea from where the problem is coming from? Thank you for helping me!

  • Optuna version: 2.9.1
  • Python version: 3.8.11
  • OS: Linux-5.11.0-38-generic-x86_64-with-glibc2.17

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:19 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
not522commented, Nov 1, 2021

Here is the speed benchmark result. https://github.com/optuna/optuna/runs/4062350310?check_suite_focus=true#step:6:17 The slowing down ratio (inmemory, random, 1000 v.s. sqlite, random, 1000) is about x50 in the GitHub Actions’ environment.

For future development, please let me know your request? 1000 trials in 1 second?

1reaction
nzw0301commented, Oct 28, 2021

Thank you for trying the code! I hope the optimisation with Redis performs not too slowly.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Data Storage Full: Slows Down Computer? (All the Info)
Here's everything about your computer slowing down if your data storage (HDD, SDD, NVMe) is full. Learn about how full internal storage ......
Read more >
Does Filling Up An SSD Slow It Down? How To Fix It!
SSDs are fast due to continuously mapping empty sectors for writing new data, eliminating the need to “erase” things, as it always writes...
Read more >
Why do programs get slower with time? - Johnny's Software Lab
This means that after you introduce the new component, completely unrelated components will be running slower because of the increase in memory ......
Read more >
6 Reasons Your PC is Slow and How to Fix It - NeweggBusiness
PCs can slow down when the storage drive is full to the brim. Your PC needs a certain amount of available space in...
Read more >
Slow Queries? Move Fast to Fix Them - Orange Matter
When limited resources like CPU or I/O run out, everything can start to slow down. Inefficient use of resources is also a problem...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found