Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Logging Precision Recall Multiclass Labeling issue

See original GitHub issue

Weights and Biases version: 0.9.5
Python version: 3.7.7
Operating System: Ubuntu

Description

My multiclass classification model produces a matrix with 3 columns to which it I put three items as its labels. However, when I try to log the model’s precision and recall I receive an error saying the labels are mismatched.

What I Did

y_probas = model.predict(X_test)
wandb.log({'pr': wandb.plots.precision_recall(y_test, y_probas, labels=["label1", "label2", "label3"])})

and I receive this traceback

Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/tensorflow2_latest_p37/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/ubuntu/anaconda3/envs/tensorflow2_latest_p37/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ubuntu/anaconda3/envs/tensorflow2_latest_p37/lib/python3.7/site-packages/wandb/wandb_agent.py", line 64, in _start
    function()
  File "<ipython-input-11-5e53f86ad550>", line 27, in tune
    wandb.log({'pr': wandb.plots.precision_recall(y_test, y_probas, labels=labels)})
  File "/home/ubuntu/anaconda3/envs/tensorflow2_latest_p37/lib/python3.7/site-packages/wandb/plots/precision_recall.py", line 46, in precision_recall
    binarized_y_true = scikit.preprocessing.label_binarize(y_true, classes=classes)
  File "/home/ubuntu/anaconda3/envs/tensorflow2_latest_p37/lib/python3.7/site-packages/sklearn/preprocessing/_label.py", line 650, in label_binarize
    .format(classes, unique_labels(y)))
ValueError: classes [0. 1.] mismatch with the labels [0 1 2] found in the data

Issue Analytics

State:
Created 3 years ago
Comments:9 (2 by maintainers)

Top GitHub Comments

1reaction

lavanyashuklacommented, Aug 21, 2020

Hi Ajay,

PR curves are usually used to analyze results from binary classification problems.

In order to extend it to a multi-class problem, please binarize the output first. You can read more about how to do so in the scikit docs.

Let me know if have more questions!

1reaction

issue-label-bot[bot]commented, Aug 18, 2020

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.89. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

Top Results From Across the Web

Logging Precision Recall Multiclass Labeling issue #1207

My multiclass classification model produces a matrix with 3 columns to which it I put three items as its labels. However, when I...

How to compute precision/recall for multiclass-multilabel ...

Precision = 1n∑ni=1|Yi∩h(xi)||h(xi)| , The ratio of how much of the predicted is correct. The numerator finds how many labels in the predicted ......

Precision/recall for multiclass-multilabel classification

Let us assume that we have a 3-class multi classification problem with labels A, B and C.

Multi Label Model Evaluation | Kaggle

Precision-Recall is a useful measure of success of prediction when the classes are very imbalanced. In information retrieval, precision is a measure of...

Comprehensive Guide on Multiclass Classification Metrics

Precision and recall for multiclass classification. Precision answers the question of “what proportion of predicted positives are truly ...