Support for float32 in `KDTree` and `BallTree`

Description

Relates to #7059 and #11000.

The conversion from float32 to float64 in Isomap leads to cython class BinaryTree

Steps/Code to Reproduce

Debug example:

from sklearn.datasets import load_digits
from sklearn.manifold import Isomap
X, _ = load_digits(return_X_y=True)
X = X.astype('float32')
embedding = Isomap(n_components=2)
X_transformed = embedding.fit_transform(X[:100])

The change of data type occurs in line 157 of isomap when calling function:

kng = kneighbors_graph(self.nbrs_, self.n_neighbors,
                               metric=self.metric, p=self.p,
                               metric_params=self.metric_params,
                               mode='distance', n_jobs=self.n_jobs)

which leads to class KNeighborsMixin in _base.py, method kneighbors (line 531):

chunked_results = Parallel(n_jobs, **parallel_kwargs)(
                delayed_query(
                    self._tree, X[s], n_neighbors, return_distance)
                for s in gen_even_slices(X.shape[0], n_jobs)
            )

here, X is dtype float32, chunked_results is a list of arrays dtype float64

The problem arises in the method query of class BinaryTree in _binary_tree.pxi (line 1271). Data are casted into either DTYPE or DTYPE_t, which are defined as np.float64.

class BinaryTree should be changed to allow computations in both float32 and float64 data types

[OmarManzoor]

Hi @OmarManzoor,

Thanks for you interest!

As stated by Gaël in #15474 (comment) and as explained in #7059 (comment), Tempita might be the best solution to introduce support for float32 in KDTree and in BallTree.

This is not the best good first issue to get started with Cython on scikit-learn.

What is your experience with Cython so far, @OmarManzoor?

I think resolving C and C++ compilations warnings of C and C++ sources file generated by Cython implementation is a good first Cython issue. I need to file an issue for describing this, but you can start to have a look at this in the meantime. Do you think this is a good starting starting point for you?

Thank you very much for such a detailed reply! To be honest I don’t have any prior experience with Cython but I am interested in learning and gaining some experience with it. I trust your judgement and would be fine with resolving compilation warnings.

[GaelVaroquaux]

Good investigation job! You can explore if cython’s fused type enable to solve this. If not, you’ll have to resort to templates, but that’s more work and less elegant.

Thanks for doing this!