Make SparseArray an ExtensionArray

We should make SparseArray a proper ExtensionArray.

It seems like this will be somewhat difficult to do properly when SparseArray subclasses ndarray. Basic things like np.asarray(sparse_array) don’t match the required ExtensionArray API (https://github.com/pandas-dev/pandas/issues/14167). Fixing this, especially when we subclass ndarray, is going to be difficult. I can’t override the behavior of np.asarray(sparse_array) in Python.

So, some questions

1. Do people rely on SparseArray being an ndarray subclass?
2. Do we want to make a clean break, or introduce deprecations for things that will need changing (but with no clear upgrade path)?

My current preference is to just break things, but I don’t use sparse. SparseArray would compose an ndarray of dense values and a SparseIndex, but it would no longer subclass ndarray.

CCing some people who seem to use pandas’ sparse: @hexgnu @kernc @Licht-T

[TomAugspurger]

Yeah, they’re not in the official public API, but they were likely not considered in the pandas.core privatization shuffle.

So, I don’t really know. I suppose it depends on if people find them useful (@hexgnu @kernc @Licht-T), otherwise I would default to making them private implementation details of SparseArray.

[jorisvandenbossche]

That let me wonder: should this be public? Or more in general, are the sparse index objects considered public? (you can pass it in the SparseSeries constructor currently, but the objects are never used in the docs / not exposed top-level).