Make suggest API more consistent and improve documentation

Motivation

We want to make behaviors of suggesting API for integer parameters and floating-point parameters consistent. Our focuses are summarized in the description.

Description

1. Suggest APIs variant for floating-point

There are four methods available for the current suggest API for a floating-point parameter: suggest_float, suggest_uniform, suggest_loguniform, and suggest_discrete_uniform. On the other hand, there is one method for each integer and categorical parameter: suggest_int and suggest_categorical. We plan to unify methods for floating point parameter into suggest_float and deprecate suggest_uniform, suggest_loguniform, and suggest_discrete_uniform. Then, we can change the implementation of suggest_uniform, suggest_loguniform, and suggest_discrete_uniform so that they are wrapped by suggest_float.

2. Documentation about behavior about the upper endpoint of floating-point distribution

While suggest API for an integer parameter would return the upper endpoint of the parameter space, suggest API for a float parameter would not return the upper endpoint according to documentation. Although it is true for some situations (e.g. TPESampler), samplers could return an upper endpoint due to numerical error of a floating-point. We decided to update our documentation that an upper endpoint would be included as a result of suggest API for float.

3. Allowing suggest_float to add a parameter during optimization (to make consistent with suggest_int)

In the current implementation, behavior is different between integer and float when adding a parameter (e.g. step) to suggest API during an optimization process. An error occurs for float while it works in the integer. To make the behavior consistent, we plan to change suggest_float so that it works when adding an additional parameter during optimization.

[HideakiImamura]

[note] This issue will be closed in #3113.

[himkt]

@xadrianzetx Right, let’s go on to the final step! 🚀