Error if using dask_ml.preprocessing RobustScaler

I want to use RobustScaler and got following error:

Traceback (most recent call last):
  File "<input>", line 14, in <module>
  File "/home/magehex/anaconda3/envs/reinbo/lib/python3.7/site-packages/sklearn/base.py", line 556, in fit_transform
    return self.fit(X, y, **fit_params).transform(X)
  File "/home/magehex/anaconda3/envs/reinbo/lib/python3.7/site-packages/dask_ml/preprocessing/data.py", line 180, in fit
    quantiles = [da.percentile(col, [q_min, 50.0, q_max]) for col in X.T]
  File "/home/magehex/anaconda3/envs/reinbo/lib/python3.7/site-packages/dask_ml/preprocessing/data.py", line 180, in <listcomp>
    quantiles = [da.percentile(col, [q_min, 50.0, q_max]) for col in X.T]
  File "/home/magehex/anaconda3/envs/reinbo/lib/python3.7/site-packages/dask/array/percentile.py", line 133, in percentile
    for i, key in enumerate(a.__dask_keys__())
AttributeError: 'numpy.ndarray' object has no attribute '__dask_keys__'

Code:

import numpy as np
from dask_ml.preprocessing import RobustScaler
import time
from sklearn.datasets.samples_generator import make_blobs

# generate 2d classification dataset
X, y = make_blobs(n_samples=100000, centers=2, n_features=380)

X_train = X.astype(np.float32)
y_train = y.astype(np.int32)

start = time.time()

RobustScaler().fit_transform(X_train,y_train)

end = time.time()
print(end - start)

[stsievert]

as a user, what’s less surprising? [choices]

I think both are surprising. The results don’t match Scikit-learn’s results in either case.

I like the approach in https://github.com/dask/dask-ml/pull/580#issuecomment-549077034: raise an error saying “use Scikit-learn. If you have to, you can wrap in a Dask array, but that’ll yield slower/approximate results”: https://github.com/dask/dask-ml/blob/7793962d990a4b5290a5cfd1aee1d5b48828f1a1/dask_ml/decomposition/pca.py#L18-L24

That presents some challenges for pipelines with Dask-ML estimators and Dask array inputs. What if RobustScaler is an element in that pipeline? Maybe there should be a wrapper class to take Dask inputs, call the corresponding Scikit-learn estimator with a NumPy array and return a Dask output?

[TomAugspurger]

So, as a user, what’s less surprising?

1. A NumPy array giving a different answer to the same Dask array with the block?
2. Our estimator giving a different output to sklearns on the same numpy array?