Add 32 bit support to neural_network module

Related to https://github.com/scikit-learn/scikit-learn/issues/11000 it would be good to add support for 32 bit computations for estimators in the neural_network module. This was done for BernoulliRBM in https://github.com/scikit-learn/scikit-learn/pull/16352 Because performance is bound by the dot product this is going to have a large impact (cf https://github.com/scikit-learn/scikit-learn/pull/17641#issuecomment-646879638)

I would even argue that unlike other models, it could make sense to add a dtype=np.float32 parameter and make calculations in 32 bit by default regardless of X.dtypes. We could also consider supporting dtype=np.float16.

[postmalloc]

Sure @rth, will open a PR in a bit.

[rth]

Great! I was hoping that it would be a bit more significant, I but I guess it also depends on the CPU vectorization support.

Maybe let’s not add the dtype parameter after all, but rather do as in https://github.com/scikit-learn/scikit-learn/pull/16352: run,

X = check_array(X, ..., dtype=(np.float64, np.float32))

first and then use X.dtype for the creation of new arrays. That would probably be more consistent with other estimators. Would you like to make a PR?