trial.suggest_uniform sampling very non-uniform

Problem

Using a call like: colsample_bytree = trial.suggest_uniform('colsample_bytree', 0.1, 0.9) produces a highly skewed distribution:

Expected behavior

The distribution should be closer to uniform

Environment

• Optuna version:

optuna.__version__  
Out[47]: '0.19.0'

• Python version:

python                    3.6.9                h5500b2f_0

• OS:

OS Name             : Microsoft Windows 10 Home
OS Version          : 10.0.18362 N/A Build 18362
OS Manufacturer     : Microsoft Corporation
OS Configuration    : Standalone Workstation
OS Build Type       : Multiprocessor Free
System Boot Time    : 1/15/2020, 5:34:36 PM
System Manufacturer : LENOVO
System Model        : 20349
System Type         : x64-based PC
System Directory    : C:\Windows\system32
System Locale       : en-us;English (United States)
Hotfix(s)           : 11 Hotfix(s) Installed.,[01]: KB4532938,[02]: KB4497165,[03]: KB4498523,[04]: KB4503308,[05]: KB4515383,[06]: KB4516115,[07]: KB4520390,[08]: KB4521863,[09]: KB4524569,[10]:
                      KB4528759,[11]: KB4528760

• (Optional) Other libraries and their versions:

xgb.__version__
Out[50]: '0.90'

Error messages, stack traces, or logs

None

Steps to reproduce

def train_model(trial,
                train_X,
                train_Y,
                val_X,
                val_Y,
                n_iters,
                params,
                n_class = None):
    colsample_bytree = trial.suggest_uniform('colsample_bytree', 0.1, 0.9)

[SBrandeis]

The default sampler for a study is TPESampler. The following might solve your issue, by changing sampler to RandomSampler (based on numpy.random):

from optuna.samplers.random import RandomSampler
seed = 11

study = optuna.create_study(sampler=RandomSampler(seed))

This yields the following distribution:

[toshihikoyanase]

@eafpres Thank you for your question. As suggested by @SBrandeis , that is the expected behavior of the default sampler. TPESampler tries to sample new parameter values from promising areas in order to get optimal parameter values. As a result, the distribution of suggested values will be skewed. Please refer to this document for further details of TPESampler.

By the way, I feel that the name of suggest_uniform is a bit confusing. IMO, renaming it to the name which is consistent with the behavior can be helpful for users.

@SBrandeis Thank you for your prompt reply!