question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

What to do with missing values in heatmaps

See original GitHub issue

Running this snippet:

import seaborn as sns
import numpy as np
import pandas as pd

df = pd.DataFrame(data={'a': [1, 1, 1],
                        'b': [2, np.nan, 2],
                        'c': [3, 3, np.nan]})

sns.heatmap(df)

We get something like this:

missing-get-small

Missing values are happily assigned “the color of the minimum” in the color bar. I wonder if this is a good default or if missing values should be treated differently. I would go for “treating them differently by default” and then the question would be how (maybe having a missing_color parameter and issuing a warning if the color is in the colormap/colorbar).

Issue Analytics

  • State:closed
  • Created 9 years ago
  • Reactions:8
  • Comments:17 (10 by maintainers)

github_iconTop GitHub Comments

22reactions
olgabotcommented, Nov 24, 2014

As of now, you could manually mask them,

import seaborn as sns
import numpy as np
import pandas as pd

df = pd.DataFrame(data={'a': [1, 1, 1],
                        'b': [2, np.nan, 2],
                        'c': [3, 3, np.nan]})

mask = df.isnull()
sns.heatmap(df, mask=mask)

image (edit: added image)

Is this what you suggest should happen by default?

4reactions
mwaskomcommented, Apr 25, 2015

There’s a masking argument that works pretty well, but it could be improved in at least two respects. First, it could be useful to add an option that makes masked out cells the same color as the image background (this would be especially pleasing for heat maps of upper-triangular or lower-triangular matrices).

@jm-contreras I think this already happens? Perhaps you mean something different:

with sns.axes_style("white"):
    sns.heatmap(flights_missing, cmap=cmap, mask=flights_missing.isnull())

bbvgncetu9iaaaaasuvork5cyii

Read more comments on GitHub >

github_iconTop Results From Across the Web

Visualizing Missing Data with Seaborn Heatmap and Displot
One of the ways to visualize the missing data is make a heatmap of the data coded as boolean for missing-ness. Second way...
Read more >
Using Heatmaps to Visualize Missing Values | by Brent Smart
Using Heatmaps to Visualize Missing Values · Inspecting specific rows and columns. · Previewing data frame columns. · Isolating rows with missing data....
Read more >
Use a heat map to visualize missing values in longitudinal data
A heat map (sometimes called a lasagna plot) is a better choice. Depending on the structure of your data, you might need to...
Read more >
How to Visualize Missing Data in R using a Heatmap
Missing values are generally represented by NA in a data frame. Here, we will describe how to visualize missing data in R using...
Read more >
Visualizing the patterns of missing value occurrence with Python
Method 1: seaborn.heatmap. The first method is by seaborn.heatmap . The next single-line code will visualize the location of missing values.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found