BUG Inconsistent f1_score behavior when combining label indicator input with labels attribute

Description

When using label indicator inputs for y_pred and y_true, metrics.f1_score calculates the macro average over all label-specific f-scores whenever the labels parameter includes column index 0. It should only average over the label-specific scores indicated by the labels parameter, as it does when 0 is not present in the labels parameter.

Steps/Code to Reproduce

import numpy as np
from sklearn.metrics import f1_score, precision_recall_fscore_support

y_true = np.array([[0, 1, 0, 0],
                   [1, 0, 0, 0],
                   [1, 0, 0, 0]])
y_pred = np.array([[0, 1, 0, 0],
                   [0, 0, 1, 0],
                   [0, 1, 0, 0]])

p, r, f, s = precision_recall_fscore_support(y_true, y_pred)
print(f)
print(f1_score(y_true, y_pred, labels=[0,1], average='macro'))
print(f1_score(y_true, y_pred, labels=[0,1,2], average='macro'))
print(f1_score(y_true, y_pred, labels=[1,3], average='macro'))
print(f1_score(y_true, y_pred, labels=[1,2,3], average='macro'))

Expected Results

[ 0.          0.66666667  0.          0.        ]
0.333333333333
0.222222222222
0.333333333333
0.222222222222

Actual Results

[ 0.          0.66666667  0.          0.        ]
0.166666666667
0.166666666667
0.333333333333
0.222222222222

Versions

Windows-7-6.1.7601-SP1 Python 3.5.3 |Anaconda custom (64-bit)| (default, May 15 2017, 10:43:23) [MSC v.1900 64 bit (AMD64)] NumPy 1.13.1 SciPy 0.19.0 Scikit-Learn 0.19.0

[anerishah97]

Can I take this up?

[gxyd]

I just noticed this. This actually is fixed by https://github.com/scikit-learn/scikit-learn/pull/10377 . I independently tried to fix that in that PR. I think it correctly fixes this issue as well.