BUG: searchsorted with object arrays containing nan

import sys
import numpy as np

arr = np.array([1, 2, 3, 4, 5], dtype=object)
arr[::2] = np.nan

print(arr)
# [nan 2 nan 4 nan]

bins = np.array([1, 3, 5])

# Inserts into same position (incorrect)
print(bins.searchsorted(arr))                                                                                                        
# array([0, 1, 0, 1, 0])

# Now inserts into different positions (correct)
print(bins.searchsorted(arr.astype(float)))
# array([3, 1, 3, 2, 3])

np.__version__
# '1.18.1'

sys.version
# '3.7.4 (default, Aug 13 2019, 15:17:50) \n[Clang 4.0.1 (tags/RELEASE_401/final)]'

In the example here searchsorted assigns both 2 and 4 from arr to the same index within bins, but 2 should go to 1 and 4 to 2. The output is correct when we cast bins to float instead of object. Also, for whatever reason I can only reproduce this issue when the NaN values are evenly spaced.

[seberg]

Ah sorry, I misread. The bins are sorted and have no NaN, so except for the NaN values the result should stay the same:

arr = np.array([1, 2, 3, 4, 5, 6, 7], dtype=object)
arr[::2] = np.nan
bins = np.array([1, 3, 5])
print("Should return identical results, since only the keys change:")
print(bins.searchsorted(arr)[1::2])
print(bins.searchsorted(arr[1::2]))

The insert place of the NaN is ill-defined, but for the other values it should probably not be. This is probably an optimization in the search (i.e. to make searching for close values faster). Maybe this optimization should be disabled for object arrays.

[Qiyu8]

if NaN can not check explicitly, maybe the only way to make the result more accurate is to remove optimization algorithm in the search. at least, we can enable the number object get the correct index. for example.

arr = np.array([np.nan, 2, np.nan, 4, np.nan], dtype=object)
bins = np.array([1, 3, 5])
print(bins.searchsorted(arr)) 
# array([0, 1, 0, 2, 0])