Use scipy.special.expit for the inverse of the logit function

In a few places in the code the expit function is manually implemented. For instance,

• sklearn/linear_model/base.py:_predict_proba_lr

  prob *= -1
  np.exp(prob, prob)
  prob += 1
  np.reciprocal(prob, prob)
  

• `sklearn/discriminant_analysis.py:predict_proba:L521

• sklearn/utils/tests/test_extmath.py
  447:        return np.log(1 / (1 + np.exp(-x)))
  

• (maybe other places I missed)

It might be better to use scipy.special.expit instead which is simpler and also appears to be faster,

In [3]: from scipy.special import expit

In [4]: import numpy as np

In [14]: y0 = np.random.rand((1000))                                                                                                                         

In [15]: %%timeit 
    ...:  
    ...: y = y0.copy() 
    ...: y *= -1 
    ...: np.exp(y, y) 
    ...: y += 1 
    ...: np.reciprocal(y, y) 
    ...:  
    ...:                                                                                                                                                     
24.8 µs ± 123 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

In [16]: %timeit y0.copy()                                                                                                                                   
544 ns ± 2.05 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [17]: %timeit expit(y)                                                                                                                                    
18 µs ± 16.1 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

@ZaydH would you be interesting in taking this, following your PR https://github.com/scikit-learn/scikit-learn/pull/12909?

[ZaydH]

I would be happy to take it too depending of @qdeffense 's preference. This was my first time working on a project of this scale so it was a good experience for me. I am happy to defer either way.

[RonLek]

Apologies @rth @qdeffense @ZaydH . It was my first time working on such a big repo. I’ll keep that in mind. Thank you.