question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BUG: skipna parameter in series.any() returns wrong result

See original GitHub issue
#importing pandas module 
import pandas as pd 

#importing numpy module
import numpy as np

data=pd.DataFrame({'A':[1,2,3,4,0,np.nan,3],
                  'B':[3,1,4,5,0,np.nan,5]})

data.any(axis=1,skipna=True)

Expected output: 0 True 1 True 2 True 3 True 4 False 5 True 6 True dtype: bool

Returned output:

0 True 1 True 2 True 3 True 4 False 5 False 6 True dtype: bool

As written in documentation, If an entire row/column is NA, the result will be NA But NA isn’t returned in any of the cases (Keeping skipna True or False)

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:6 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
jorisvandenbosschecommented, Oct 15, 2018

Skipna should be the same as the operation on the values with NAs removed (is that right @jorisvandenbossche?).

I suppose this as well. any/all can be seen as reductions like sum or prod, so we should probably follow their design.

So I think @dsaxton is right that it is only the documentation that is incorrect.

1reaction
dsaxtoncommented, Oct 14, 2018

My opinion: I think the problem is the documentation; the result is actually correct. If you ask if any of an empty set of statements is True, the answer is no. This is consistent with numpy:

In [1]: import numpy as np

In [2]: np.any([])
Out[2]: False
Read more comments on GitHub >

github_iconTop Results From Across the Web

Pandas: Use .resample() and .mean() with skipna=False
I'd like to bucket my dataframe by second and take the mean, whereby if there exist nan values in any bucket, I'd like...
Read more >
Not-So-Trivial Behavior of the df.sum() Pandas Method in the ...
In this article, we'll take a look at some… ... assigning the skipna parameter to False and restoring the normal "pythonic" behavior of...
Read more >
pandas.Series — pandas 1.5.2 documentation
Invoke function on values of Series. argmax ([axis, skipna]). Return int position of the largest value in the Series.
Read more >
pandas.DataFrame.mean() Examples
This by default returns a Series, if level specified, ... for columns use column/1; skipna – Excludes all None/NaN from teh mean result....
Read more >
Pandas DataFrame any() Method - W3Schools
The any() method returns one value for each column, True if ANY value in that ... The axis , bool_only , skipna ,...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found