question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Matthews correlation coefficient metric throws misleading division by zero RuntimeWarning

See original GitHub issue

Description

With tested values all equal, sklearn.metrics.matthews_corrcoef throws a RuntimeWarning reporting a division by zero. This behavior was already reported in #1937 and reported fixed, but reappears in recent versions.

Steps/Code to Reproduce

The snippet below reproduces the warning.

import sklearn.metrics                         
trues = [1,0,1,1,0]                            
preds = [0,0,0,0,0]                            
sklearn.metrics.matthews_corrcoef(trues, preds)

Expected Results

No warning is thrown.

Actual Results

The following warning is thrown:

C:\anaconda\envs\sklearn-test\lib\site-packages\sklearn\metrics\_classification.py:900: RuntimeWarning: invalid value encountered in double_scalars
  mcc = cov_ytyp / np.sqrt(cov_ytyt * cov_ypyp)

Versions

System:
    python: 3.8.2 (default, Mar 25 2020, 08:56:29) [MSC v.1916 64 bit (AMD64)]
executable: C:\anaconda\envs\sklearn-test\python.exe
   machine: Windows-10-10.0.18362-SP0

Python dependencies:
       pip: 20.0.2
setuptools: 46.1.3.post20200330
   sklearn: 0.22.1
     numpy: 1.18.1
     scipy: 1.4.1
    Cython: None
    pandas: None
matplotlib: None
    joblib: 0.14.1

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:5
  • Comments:7 (5 by maintainers)

github_iconTop GitHub Comments

4reactions
Connerrrrrcommented, Apr 25, 2021

@jnothman PR was created.

0reactions
Connerrrrrcommented, Apr 1, 2022

Was this solved in 2022? I’m still getting this error RuntimeWarning: invalid value encountered in double_scalars mcc = cov_ytyp / np.sqrt(cov_ytyt * cov_ypyp)

@scienception Any guidance of reproduce the error?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Warning that seems a bug in Matthews correlation coefficient ...
The warning is caused by a division-by-zero due to no variance in your predictions. It is misleading and can be ignored; the metric...
Read more >
Matthews Correlation Coefficient - a Hugging Face Space by ...
Compute the Matthews correlation coefficient (MCC) The Matthews correlation coefficient is used in machine learning as a measure of the quality of binary ......
Read more >
The Matthews correlation coefficient (MCC) is more reliable ...
A Matthews correlation coefficient close to +1, in fact, means having high values for all the other confusion matrix metrics.
Read more >
BERT Fine-Tuning Tutorial with PyTorch - Chris McCormick
Divide up our training set to use 90% for training and 10% for ... Matthew's correlation coefficient because this is the metric used...
Read more >
SciPy 0.19.0 Release Notes — SciPy v1.9.3 Manual
The underlying LAPACK routines are replaced with “expert” versions and now can also be used to solve symmetric, hermitian and positive definite coefficient...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found