SelectFromModel threshold for ElasticNet is wrong

Describe the bug

As an estimator with an L1 penalty, ElasticNet should default to 1e-5 as the selection threshold. SelectFromModel code does not account for this, since ElasticNet has no penalty parameter, and is not called LASSO.

https://github.com/scikit-learn/scikit-learn/blob/80598905e517759b4696c74ecc35c6e2eb508cff/sklearn/feature_selection/_from_model.py#L22-L31

Steps/Code to Reproduce

from sklearn.feature_selection import SelectFromModel from sklearn.linear_model import ElasticNet from sklearn.datasets import make_regression

en = ElasticNet() fs = SelectFromModel(estimator=en)

X, y = make_regression(n_samples=100, n_features=10, n_informative=3, random_state=42)

fs_fit = fs.fit(X=X, y=y)

print(fs_fit.threshold_)

Expected Results

1e-5

Actual Results

6.53140663228231

Versions

System:
    python: 3.9.10 | packaged by conda-forge | (main, Feb  1 2022, 21:24:11)  [GCC 9.4.0]
executable: /opt/conda/bin/python
   machine: XXXXXXXX

Python dependencies:
      sklearn: 1.1.1
          pip: 22.0.3
   setuptools: 60.9.3
        numpy: 1.21.6
        scipy: 1.8.0
       Cython: 0.29.28
       pandas: 1.4.2
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libgomp
       filepath: /opt/conda/lib/python3.9/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
    num_threads: 16

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /opt/conda/lib/libopenblasp-r0.3.20.so
        version: 0.3.20
threading_layer: pthreads
   architecture: XXXXXX
    num_threads: XXXXXX

[jerryqhyu]

@haochunchang it is here

https://github.com/scikit-learn/scikit-learn/blob/80598905e517759b4696c74ecc35c6e2eb508cff/sklearn/linear_model/_coordinate_descent.py#L698-L701

[haochunchang]

Hi, in the hope to facilitate the discussion, I opened a PR to add the special cases for ElasticNet. By the way, I am curious where can I find the documentation says:

Currently, l1_ratio <= 0.01 is not reliable, unless you supply your own sequence of alpha

Thanks!