Confusion Matrix Representation / Return Value

Describe the workflow you want to enable

An enhancement to the output of confusion matrix function, better representing the true and predicted values for multilevel classes.

• i.e. Current Representation with code: from sklearn.metrics import confusion_matrix y_true = ["cat", "ant", "cat", "cat", "ant", "bird"] y_pred = ["ant", "ant", "cat", "cat", "ant", "cat"] confusion_matrix(y_true, y_pred, labels=["ant", "bird", "cat"]) Output: array([[2, 0, 0], [0, 0, 1], [1, 0, 2]])

Describe your proposed solution

When you have multiple levels you can have difficulty reading the ndarray, associating the levels with the True and Predicted values.

Proposed solution should look similar to the table below, providing better readability of the confusion matrix.

		Predicted	Value
	Levels	ant	bird	cat
True	ant	2	0	0
Value	bird	0	0	1
	cat	1	0	2

Possible Solutions:

1. Provide a parameter to prettyprint the matrix. printMatrix [type:bool]
2. Include another parameter to return ndarray, index as true_values, columns as predicted_values For example: cm = confusion_matrix(y_true, y_pred, labels=["ant", "bird", "cat"]) index=["true:ant", "true:bird", "true:cat"] columns=["pred:ant", "pred:bird", "pred:cat"] return cm, index, columns Which can be easily converted into a dataframe for further use

[shubhamdo]

Returning tuples as keys within the nested dict work well with casting to DataFrame, please refer below dict structure representing the same.

{('pred', 'ant'): {('true', 'ant'): 2, ('true', 'bird'): 0, ('true', 'cat'): 1},
('pred', 'bird'): {('true', 'ant'): 0, ('true', 'bird'): 0, ('true', 'cat'): 0}, 
('pred', 'cat'): {('true', 'ant'): 0, ('true', 'bird'): 1, ('true', 'cat'): 2}}

Screenshot:

Another Option: Flat Dict Here, I think the problem with flat dict option is the casting the dataframe will need extra steps. If we need one step conversion then the nested dict with tuples as keys is good as seen above.

Flat Dict:

{('pred_ant', 'true_ant'): 2, ('pred_ant', 'true_bird'): 0, ('pred_ant', 'true_cat'): 1,
 ('pred_bird', 'true_ant'): 0, ('pred_bird', 'true_bird'): 0, ('pred_bird', 'true_cat'): 0,
 ('pred_cat', 'true_ant'): 0, ('pred_cat', 'true_bird'): 1, ('pred_cat', 'true_cat'): 2}

Screenshot:

I think nested dict with tuples as key represent the data very well preserving class names and easy conversion to dataframe. Please let me know your thoughts, I’ll make the changes accordingly. Thanks!

[shubhamdo]

@jnothman @glemaitre taking your comments in due consideration, I’ve added another change so that it returns an output in a default data type i.e. dict() Eliminating need for hard dependencies, increasing usability with any other 3rd party libs too.

Please refer to the example code and screenshots. Thanks!

Example: from sklearn.metrics._classification import confusion_matrix y_true = [“cat”, “ant”, “cat”, “cat”, “ant”, “bird”] y_pred = [“ant”, “ant”, “cat”, “cat”, “ant”, “cat”] cm = confusion_matrix(y_true, y_pred, labels=[“ant”, “bird”, “cat”], pprint=True)

Code: Output as a dict()

    if pprint:
        labelList = labels.tolist()

        cm_lol = cm.tolist()
        cm_dict = {str(labelList[j]): {str(labelList[i]): cm_lol[i][j] for i in
                                                 range(0, len(labelList))} for j in range(0, len(cm_lol))}

        return cm_dict

Output w/o pprint(False):

array([[2, 0, 0],
       [0, 0, 1],
       [1, 0, 2]], dtype=int64)

Output w/ pprint(True):

{'ant': {'ant': 2, 'bird': 0, 'cat': 1}, 
'bird': {'ant': 0, 'bird': 0, 'cat': 0},
 'cat': {'ant': 0, 'bird': 1, 'cat': 2}} 

For this solution changes required:

def confusion_matrix(y_true, y_pred, *, labels=None, sample_weight=None,
                     normalize=None, pprint=False):
....... 
 if pprint: # Logic
      return cm_dict #As the suggested output
.....

return cm

Option 2: For better understanding the true and pred values.

    if pprint:
        labelList = labels.tolist()

        cm_lol = cm.tolist()
        cm_dict = {"pred_" + str(labelList[j]): {"true_" + str(labelList[i]): cm_lol[i][j] for i in
                                                 range(0, len(labelList))} for j in range(0, len(cm_lol))}

        return cm_dict

Output:

{'pred_ant': {'true_ant': 2, 'true_bird': 0, 'true_cat': 1},
 'pred_bird': {'true_ant': 0, 'true_bird': 0, 'true_cat': 0}, 
'pred_cat': {'true_ant': 0, 'true_bird': 1, 'true_cat': 2}}