LogisticRegression spends a lot of time in exp and log1p

Something like 200s in comparison to the 13s spent in .dot / matvec. This is from looking at the profiler after running through the dask-glm example notebook.

Is it normal to spend so much time in scalar functions? Are there approximations that make sense to do?

This is low priority from my perspective (there is plenty more to do) but I thought I’d record it.

[stsievert]

I don’t think I’d expect this imbalance, but it definitely depends on the number of times each function is called in one training iteration. It looks like we should expect an equal number of calls to exp, log and dot (at least after a brief look at the source).

The motivation for SGD is when the number of examples is much larger than the features for each example. That would be one way to reduce the time for exp/log/dot, but I don’t think it’s directly related to this issue (though it seems relevant). I can look into this.

I’ve done some brief timing with datasets of the same size as the taxicab:

[ogrisel]

Interpreting the parameters of a severely underfitting model is probably misleading 😉

Also interpreting polynomial features is fine (if the degree is 2 or 3). You can threshold the small weights of a first linear model trained on the full polynomial feature expansion to keep the top 100 or so and refit only on those and interpret them.