Un-deprecate randomized_l1?

The users are revolting at #8995 😃

Do we want to:

1. keep it deprecated?
2. remove the deprecation (until further notice)?
3. re-implement it with a different interface, which determines feature_importances_ and can be wrapped in SelectFromModel?
4. review potential improvements to the method?

[GaelVaroquaux]

You mean: 3 users are complaining. Sure. Every time we change something, some people will find that the change that not suit them.

I still think that we should deprecate. If people really want it, they can do a contrib package.

The randomized l1 models are just not reliable. They are interesting from a theoretical perspective, but they are unstable and have too many hyper parameters. They are a cost to us, and not a strong benefit to users.

As I mentioned in #8995, I just don’t know of any replacement to do feature selection in high-dimensions. This is just something that it not solved. The fact that there are so many papers but that none of them is used reliably in application tells us something. And before anyone mentions genomics, let me stress that to my knowledge, careful evaluation of sparse methods in genomics [1] has shown that a simple anova (aka T-test) works better.

[1] http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0028210

[cruyffturn]

You mean: 3 users are complaining. Sure. Every time we change something, some people will find that the change that not suit them.

I still think that we should deprecate. If people really want it, they can do a contrib package.

The randomized l1 models are just not reliable. They are interesting from a theoretical perspective, but they are unstable and have too many hyper parameters. They are a cost to us, and not a strong benefit to users.

As I mentioned in #8995, I just don’t know of any replacement to do feature selection in high-dimensions. This is just something that it not solved. The fact that there are so many papers but that none of them is used reliably in application tells us something. And before anyone mentions genomics, let me stress that to my knowledge, careful evaluation of sparse methods in genomics [1] has shown that a simple anova (aka T-test) works better.

[1] http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0028210

@GaelVaroquaux In my understanding, the reference paper only applies randomization across the samples. While the randomized lasso also scales the columns randomly. So the paper doesn’t exactly show randomized lasso is inferior.