NDCG score doesn't work with binary relevance and a list of 1 element

See this code example:

>>> t = [[1]]
>>> p = [[0]]
>>> metrics.ndcg_score(t, p)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/cbournhonesque/.pyenv/versions/bento/lib/python3.8/site-packages/sklearn/utils/validation.py", line 63, in inner_f
    return f(*args, **kwargs)
  File "/Users/cbournhonesque/.pyenv/versions/bento/lib/python3.8/site-packages/sklearn/metrics/_ranking.py", line 1567, in ndcg_score
    _check_dcg_target_type(y_true)
  File "/Users/cbournhonesque/.pyenv/versions/bento/lib/python3.8/site-packages/sklearn/metrics/_ranking.py", line 1307, in _check_dcg_target_type
    raise ValueError(
ValueError: Only ('multilabel-indicator', 'continuous-multioutput', 'multiclass-multioutput') formats are supported. Got binary instead

It works correctly when the number of elements is bigger than 1: https://stackoverflow.com/questions/64303839/how-to-calculate-ndcg-with-binary-relevances-using-sklearn

[cBournhonesque]

HI jerome, you are right, I made a mistake. I’m using the definition on wikipedia It looks like the results would be 0.0 if the document isn’t a relevant one (relevance=0), or 1.0 if it is (relevance > 0). So the returned value could be equal to y_true[0] > 0. ? In any case, I think that just updating error messages but keeping the current behaviour could be fine too

[georged4s]

Thanks @glemaitre for replying and for the heads up. Cool, I will look into this one.