Outputs vary for Koalas Series vs Pandas Series when we have np.nan
See original GitHub issueThe behavior of the Series functions changes when I introduced a np.nan in the series. But Pandas seem to handle this just fine. Some of the functions and their outputs.
def usingSeries():
s = ks.Series([11, 3, 5, 6, 8,np.nan])
print(s.max())
print(s.min())
print(s.kurtosis())
print(s.std())
nan
3.0
nan
nan
def usingPdSeries():
s = pd.Series([11, 3, 5, np.nan, 6, 8])
print(s.max())
print(s.min())
print(s.kurtosis())
print(s.std())
11.0
3.0
-0.0034686090877542597
3.049590136395381
Issue Analytics
- State:
- Created 4 years ago
- Comments:5 (3 by maintainers)
Top Results From Across the Web
How to convert np.where() while converting pandas to koalas?
The code is working if I convert the koalas back to pandas using toPandas() or from_pandas() , but due to performance and scalability...
Read more >Series — Koalas 1.8.2 documentation - Read the Docs
Conform Series to new index with optional filling logic, placing NA/NaN in locations having no value in the previous index. Series.reindex_like (other).
Read more >10 Minutes from pandas to Koalas on Apache Spark - Databricks
A Koalas Series can be created by passing a list of values, the same way as a pandas Series. A Koalas Series can...
Read more >16 Underrated Pandas Series Methods And When To Use Them
In this article, we're going to explore some lesser-known but very useful pandas methods for manipulating Series objects.
Read more >Koalas: Easy Transition between pandas and Spark - Medium
If you are a data scientist you might have used pandas data-frame API to get your ... Output comparison between Pandas and Koalas...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
When running
with Koalas 0.2, I get the following output:
Has this issue been fixed with the latest Koalas release or is something wrong with @akhilanandbv003’s setup?
It works fine for me.