question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Incorrect results for not (NaN between A and B)

See original GitHub issue

Consider a table with a double column with some NaN values.The following queries return inconsistent results. not (val between 1 and 3) when used as a filter excludes NaN rows, but when used as a projection returns true.

presto:tpch> select * from nan;
 val
-----
 NaN
(1 row)

presto:tpch> select * from nan where not (val between 1 and 3);
 val
-----
(0 rows)

presto:tpch> select not (val between 1 and 3) from nan;
 _col0
-------
 true
(1 row)

Issue Analytics

  • State:open
  • Created 4 years ago
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
kaikalurcommented, Dec 5, 2019

Quick check in the spec doesn’t find NaN at all. That’s why I thought it would be just like NULL. I will read the spec more carefully and update it. We can look at Java semantics I suppose.

1reaction
kaikalurcommented, Dec 5, 2019

OK. To simplify, I just made it a < and I see a somewhat related issue. Evaluating using NaN seems to be messed up:

 select * from test_sreeni_temp;
  x                   
-----
 NaN 


select not(x < 2), x < 2, not(x > 2), x > 2, not (x between 1 and 2), x between 1 and 2 from test_sreeni_temp;
 _col0 | _col1 | _col2 | _col3 | _col4 | _col5 
-------+-------+-------+-------+-------+-------
 false | false | false | false | true  | false 

so both x </> 2 and not(x </> 2) return false - not confusing at all! And to make more fun between has a completely different behavior.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Why is NaN not equal to NaN? [duplicate] - Stack Overflow
The accepted answer is 100% without question WRONG. Not halfway wrong or even slightly wrong. I fear this issue is going to confuse...
Read more >
Working with missing data — pandas 1.5.2 documentation
In [13]: df2["one"] == np.nan Out[13]: a False b False c False d False e False f False g ... pandas objects provide...
Read more >
NaN - JavaScript - MDN Web Docs
The global NaN property is a value representing Not-A-Number. ... Math operation where the result is not a real number (e.g. Math.sqrt(-1) );...
Read more >
Detect and count missing values (NaN) with isnull(), isna()
This article describes how to check if pandas.DataFrame and Series contain missing values and count the number of missing and non-missing ...
Read more >
Determine which array elements are NaN - MATLAB isnan
This MATLAB function returns a logical array containing 1 (true) where the elements of A are NaN, and 0 (false) where they are...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found