Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

study.best_trial takes minutes to load from postgres

See original GitHub issue

Expected behavior

I have a study with a few thousand trials. It takes a few minutes to simply load the best trial. It looks like not only is this query happening in the application layer, but the number of queries executed is O(n) instead of O(1).

Environment

Optuna version: 0.19.0
Python version: 3.6.8
OS: Ubuntu 18.04

Steps to reproduce

Use postgres backend
Create a study with thousands of trials
In a new process, attempt to access the studies best_trial

It takes similar amounts of time to do things like get a data frame with all trial information, render visualizations, etc.

Issue Analytics

State:
Created 4 years ago
Comments:7 (7 by maintainers)

Top GitHub Comments

1reaction

dwielcommented, Dec 3, 2019

For reference, new issues opened are #762 and #763

1reaction

dwielcommented, Dec 3, 2019

Cool, yeah looks like #729 will fix this, thanks. I can change to enhancement, but my thought when filing this issue was that this is something should take milliseconds, not minutes, and there are many applications (mine included) where this difference is the difference between usable and unusable. We aren’t talking a few percent speed up, we are talking 5 orders of magnitude difference. Either way I can close this as dup and open a new one of the enumeration of the dataframe/visualization as a separate issue.

Top Results From Across the Web

Postgresql simple query takes too much time like five minutes ...

When you use the LIMIT clause, it is applied after all the data is loaded to filter down the data, which is evident...

Postgres SQL query takes more than a minute SOMETIMES to ...

I have a Postgres database with a fairly large table for events, average query time for this table is around 1 second when...

PostgreSQL Running Slow? Tips & Tricks to Get to the Source

This blog provides tips and tricks for identifying issues in your PostgreSQL setup and how to tune it to drive performance.

100x Faster Postgres Performance by Changing 1 Line

Use these step-by-step instructions to monitor slow Postgres queries to improve Postgres performance. Learn more.

9.9. Date/Time Functions and Operators - PostgreSQL

All the functions and operators described below that take time or timestamp ... current_timestamp, timestamp with time zone, Current date and time (start...