Creation of an experimental module

The creation of an experimental module has been discussed and accepted during the recent Paris sprint. Experimental estimators / tools may break API and features without deprecation cycles.

We agreed to put the new GBDTs there so #12807 creates the experimental module, in a very minimal fashion.

However I don’t think there is consensus on how experimental tools should be imported by the users. IIRC 2 options were discussed:

Option 1:

from sklearn.ensemble import NewGBDTs  # raises a warning (or an error?):
                                       # "you're using an experimental feature"

import experimental  # silences the warning
from sklearn.ensemble import NewGBDTs  # no warning

Advantage:

• No need to change user code when NewGBDTs become stable in ensemble.
• no need for warnings “NewGBDTs will be removed from experimental and are now accessible from ensemble”

Drawback:

• there’s no way to know which estimators / tools are experimental and which aren’t just by looking at the code
• there’s a chance that users start using import experimental everywhere and then their code would break and they won’t know why

Option 2:

from sklearn.experimental import NewGBDTs

When NewGBDTs become stable, a warning is issued “NewGBDTs will be removed from experimental and are now accessible from ensemble”

Advantage:

• more explicit (you know what’s experimental and what isn’t)

Drawback:

• Users need to change their code even when the API of the estimator doesn’t change.

[jnothman]

If concerned about the meaning of enable_experimental, call it i_understand_this_is_an_experimental_api. I think the import based solution is just a nuisance when things move.

[NicolasHug]

You’re talking about option 1 Adrin right? I agree this is a serious flaw.

OK, option 2 seems to be accepted. That’s what the new GBDTs PR does.