Quantity objects with nan values get duplicated in sets

When creating a set out of multiple numpy.nan objects, it will return a set with only one numpy.nan object within it.

>>> from numpy import nan
>>> {nan, nan}
{nan}

However, if we try to create a set out of multiple Quantity objects that have the same units but values of numpy.nan, then the set will contain multiple Quantity objects. (Here I chose attojansky as the funniest unit I could think of, in lieu of #5927.)

>>> from astropy.units import attojansky
>>> {nan * attojansky, nan * attojansky}
{<Quantity nan aJy>, <Quantity nan aJy>}

I would have expected that the result be a set containing only one Quantity: {<Quantity nan aJy>}.

One possibility would be to make it so that a Quantity with a value of numpy.nan be equal to itself. However, numpy.nan does not equal itself, so this change could lead to some inconsistencies or unexpected behavior.

>>> nan == nan
False

[This also reminds me of the quote from The Prisoner…“I an NaN, I am a free man!”]

Cross-references: #5420, #5424

[MSeifert04]

Using NaN in sets/dicts is more or less undefined behavior because these data structures rely on equality (but np.nan != np.nan) but on the other hand some NaN instances reference the same instance (np.nan is np.nan). It’s not really a bug, that’s just how Python (or better CPython) works:

>>> {np.nan, np.nan}  # same instance
{nan}

>>> set(np.array([np.nan, np.nan]))  # different instances when iterated over
{nan, nan}

It essentially boils down to the implementation detail that CPython uses PyObject_RichCompareBool (which is True for same instances) in some places instead of using plain == (which isn’t True for NaNs, never).

[mhvk]

I’m not sure what to do about this, as a similar problem also occurs with ndarray:

a = np.array(nan)
{a, a}
# TypeError: unhashable type: 'numpy.ndarray'

Indeed, the above suggests that Quantity should not be hashable in the first place (since it can be changed in-place).