certain float numbers get rounded
See original GitHub issueOrientDB Version, operating system, or hardware.
- v.2.1.19 (build 2.1.x@r71509d3123205ed8c61f115a5caa5ece4b87a192; 2016-06-01 12:15:51)
- v.2.1.21 (build 2.1.x@rd6619775543ef589bb487c9bd237fc703429d172; 2016-08-10 14:33:00)
- v.2.2.6 (build 2.2.x@r4b1cc998b36ca2ae21a1679938912a8f545a994a; 2016-07-27 15:34:14)
Operating System
- Linux
- MacOSX
- Windows
- Other Unix
- Other, name?
Expected behavior and actual behavior
running query select 3.193359375
EXPECTED: 3.193359375
ACTUAL: returned rounded number: 3.1933594
Tested via REST API, Orient DB studio UI, and python binary client pyorient ver. 1.5.4 - same result on all of them.
BUT if run query select 3.193359374
i get correct float in return: 3.193359374
Here is the list of my numbers (value_in, value_out) than i stored and gotten rounded values:
[(1.529296875, 1.5292969), (-36.2109375, -36.210938), (1.08984375, 1.0898438), (-80.68359375, -80.68359), (70.13671875, 70.13672), (-13.53515625, -13.535156), (-36.03515625, -36.035156), (80.68359375, 80.68359), (-3.33984375, -3.3398438), (1.041015625, 1.0410156), (-4.74609375, -4.7460938), (1.02734375, 1.0273438), (-75.05859375, -75.05859), (-43.41796875, -43.41797), (1.724609375, 1.7246094), (1.072265625, 1.0722656), (87.71484375, 87.71484), (1.595703125, 1.5957031), (53.7890625, 53.789062), (3.791015625, 3.7910156), (1.51953125, 1.5195312), (67.32421875, 67.32422), (-24.2578125, -24.257812), (-45.3515625, -45.351562), (-83.14453125, -83.14453), (4.634765625, 4.6347656), (-36.73828125, -36.73828), (1.576171875, 1.5761719), (-81.9140625, -81.91406), (1.533203125, 1.5332031), (1.70703125, 1.7070312), (1.732421875, 1.7324219), (-24.08203125, -24.082031), (-51.50390625, -51.503906), (1.716796875, 1.7167969), (-49.39453125, -49.39453), (1.04296875, 1.0429688), (1.044921875, 1.0449219), (1.052734375, 1.0527344), (-43.2421875, -43.242188), (67.67578125, 67.67578), (1.00390625, 1.0039062), (-10.72265625, -10.722656), (76.81640625, 76.81641), (-47.28515625, -47.285156), (45.17578125, 45.17578), (1.150390625, 1.1503906), (1.05078125, 1.0507812), (-18.45703125, -18.457031), (-9.31640625, -9.316406), (-85.25390625, -85.25391), (1.599609375, 1.5996094), (-31.81640625, -31.816406), (-72.0703125, -72.07031), (-14.94140625, -14.941406), (-27.0703125, -27.070312), (-42.71484375, -42.714844), (81.73828125, 81.73828), (54.31640625, 54.316406), (4.669921875, 4.669922), (1.15234375, 1.1523438), (-43.9453125, -43.945312), (-68.5546875, -68.55469), (-5.80078125, -5.8007812), (1.14453125, 1.1445312), (1.583984375, 1.5839844), (-7.91015625, -7.9101562), (3.228515625, 3.2285156), (1.525390625, 1.5253906), (-81.03515625, -81.03516), (1.54296875, 1.5429688), (-81.2109375, -81.21094), (4.658203125, 4.658203), (-44.6484375, -44.648438), (1.443359375, 1.4433594), (1.58984375, 1.5898438), (82.08984375, 82.08984), (1.728515625, 1.7285156), (1.03515625, 1.0351562), (-88.76953125, -88.76953), (45.3515625, 45.351562), (4.365234375, 4.3652344), (1.544921875, 1.5449219), (76.46484375, 76.46484), (-6.50390625, -6.5039062), (1.060546875, 1.0605469), (-82.6171875, -82.61719), (-51.6796875, -51.679688), (-78.3984375, -78.39844), (-25.6640625, -25.664062), (-42.5390625, -42.539062), (89.47265625, 89.47266), (4.67578125, 4.6757812), (1.548828125, 1.5488281), (-52.20703125, -52.20703), (4.060546875, 4.060547), (1.033203125, 1.0332031), (59.4140625, 59.414062), (-69.43359375, -69.43359), (-48.69140625, -48.691406), (-8.96484375, -8.964844), (-5.09765625, -5.0976562), (1.73046875, 1.7304688), (89.82421875, 89.82422), (-55.72265625, -55.722656), (-53.0859375, -53.085938), (-10.01953125, -10.019531), (53.61328125, 53.61328), (-71.54296875, -71.54297), (1.001953125, 1.0019531), (53.96484375, 53.964844), (1.55859375, 1.5585938), (72.0703125, 72.07031), (1.056640625, 1.0566406), (-46.0546875, -46.054688), (-25.48828125, -25.488281), (-16.5234375, -16.523438), (1.521484375, 1.5214844), (-34.27734375, -34.277344), (77.16796875, 77.16797), (1.72265625, 1.7226562), (1.41015625, 1.4101562), (-82.44140625, -82.44141), (-16.69921875, -16.699219), (1.541015625, 1.5410156), (1.05859375, 1.0585938), (1.53515625, 1.5351562), (1.064453125, 1.0644531), (1.71484375, 1.7148438), (-46.58203125, -46.58203), (1.07421875, 1.0742188), (66.4453125, 66.44531), (-44.47265625, -44.472656), (-80.5078125, -80.50781), (61.34765625, 61.347656), (67.1484375, 67.14844), (1.52734375, 1.5273438), (3.240234375, 3.2402344), (1.572265625, 1.5722656), (1.048828125, 1.0488281), (-44.82421875, -44.82422), (-16.34765625, -16.347656), (76.11328125, 76.11328), (-34.1015625, -34.101562), (1.60546875, 1.6054688), (63.80859375, 63.808594), (1.517578125, 1.5175781), (1.537109375, 1.5371094), (-76.81640625, -76.81641), (-81.38671875, -81.38672), (-41.66015625, -41.660156), (3.404296875, 3.4042969), (1.591796875, 1.5917969), (-34.98046875, -34.98047), (-24.43359375, -24.433594), (-15.29296875, -15.292969), (-4.39453125, -4.3945312), (-63.80859375, -63.808594), (-12.48046875, -12.480469), (-89.47265625, -89.47266), (4.16015625, 4.1601562), (1.57421875, 1.5742188), (1.556640625, 1.5566406), (1.58203125, 1.5820312), (89.6484375, 89.64844), (-20.7421875, -20.742188), (-6.15234375, -6.1523438), (-39.90234375, -39.902344), (81.38671875, 81.38672), (87.36328125, 87.36328), (-29.1796875, -29.179688), (-66.62109375, -66.62109), (-82.08984375, -82.08984), (4.623046875, 4.623047), (-15.64453125, -15.644531), (3.24609375, 3.2460938), (1.087890625, 1.0878906), (1.69921875, 1.6992188), (-54.66796875, -54.66797), (-51.85546875, -51.85547), (1.705078125, 1.7050781), (3.205078125, 3.2050781), (-79.62890625, -79.62891), (-36.38671875, -36.38672), (54.4921875, 54.492188), (3.22265625, 3.2226562), (1.607421875, 1.6074219), (1.708984375, 1.7089844), (-35.33203125, -35.33203), (-43.06640625, -43.066406), (1.552734375, 1.5527344), (-35.5078125, -35.507812), (88.9453125, 88.94531), (1.029296875, 1.0292969), (-15.99609375, -15.996094), (3.216796875, 3.2167969), (1.720703125, 1.7207031), (1.083984375, 1.0839844), (53.0859375, 53.085938), (-8.61328125, -8.613281), (4.611328125, 4.611328), (-72.24609375, -72.24609), (-87.01171875, -87.01172), (-27.59765625, -27.597656), (1.603515625, 1.6035156), (1.34765625, 1.3476562), (-23.37890625, -23.378906), (-11.77734375, -11.777344), (82.79296875, 82.79297), (1.587890625, 1.5878906), (-83.84765625, -83.84766), (44.6484375, 44.648438), (1.068359375, 1.0683594), (1.373046875, 1.3730469), (-13.88671875, -13.886719), (4.62890625, 4.6289062), (1.513671875, 1.5136719), (1.580078125, 1.5800781), (1.06640625, 1.0664062), (-7.55859375, -7.5585938), (84.90234375, 84.90234), (-32.87109375, -32.871094), (56.6015625, 56.601562), (1.025390625, 1.0253906), (1.59765625, 1.5976562), (-10.37109375, -10.371094), (-45.17578125, -45.17578), (-50.2734375, -50.273438), (4.65234375, 4.6523438), (-17.05078125, -17.050781), (44.82421875, 44.82422), (-74.70703125, -74.70703), (1.55078125, 1.5507812), (4.646484375, 4.6464844), (-13.18359375, -13.183594), (1.701171875, 1.7011719), (-35.68359375, -35.683594), (1.021484375, 1.0214844), (-31.2890625, -31.289062), (-41.1328125, -41.132812), (4.306640625, 4.3066406), (69.43359375, 69.43359), (3.193359375, 3.1933594), (88.76953125, 88.76953), (1.037109375, 1.0371094), (-88.41796875, -88.41797), (-52.3828125, -52.382812), (-89.82421875, -89.82422), (1.712890625, 1.7128906), (-77.6953125, -77.69531), (1.01953125, 1.0195312)]
Issue Analytics
- State:
- Created 7 years ago
- Reactions:2
- Comments:25 (6 by maintainers)
Top Results From Across the Web
Rounding Errors - The Floating-Point Guide
Because floating-point numbers have a limited number of digits, they cannot represent all real numbers accurately: when there are more digits than the...
Read more >How to round float numbers in javascript? - Stack Overflow
var number = 6.688689; var roundedNumber = Math.round(number * 10) / 10; ... I have very good solution with if toFixed() is not...
Read more >How to round up a float number in Python? - EasyTweaks.com
Today, we'll learn how to round up float numbers using Python. Solution: We have two methods to handle the rounding float numbers in...
Read more >4 Examples to Round Floating-Point Numbers in Java up to 2 ...
You can round any floating-point numbers in Java unto n places By using either Math.round(), BigDecimal, or DecimalFormat. I personally prefer to use...
Read more >Python round() function with EXAMPLES - Guru99
Round () Round() is a built-in function available with python. It will return you a float number that will be rounded to the...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Hey I just stumbled on this so sorry if I’m mistaken, but you might be making a very common mistake using Float.MIN_VALUE. It actually isn’t the lowest negative number a Float can hold, it is the smallest positive value a Float can hold. To get the lowest negative number I think you just have to do (Float.MAX_VALUE * -1). I remember running into that before and I think that’s the solution.
@almibe Nice catch! Given the purpose here (to ensure that the data is stored with the most appropriate type), the test should probably use the absolute value of
dou
.