question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

certain float numbers get rounded

See original GitHub issue

OrientDB Version, operating system, or hardware.

  • v.2.1.19 (build 2.1.x@r71509d3123205ed8c61f115a5caa5ece4b87a192; 2016-06-01 12:15:51)
  • v.2.1.21 (build 2.1.x@rd6619775543ef589bb487c9bd237fc703429d172; 2016-08-10 14:33:00)
  • v.2.2.6 (build 2.2.x@r4b1cc998b36ca2ae21a1679938912a8f545a994a; 2016-07-27 15:34:14)

Operating System

  • Linux
  • MacOSX
  • Windows
  • Other Unix
  • Other, name?

Expected behavior and actual behavior

running query select 3.193359375

EXPECTED: 3.193359375 ACTUAL: returned rounded number: 3.1933594 Tested via REST API, Orient DB studio UI, and python binary client pyorient ver. 1.5.4 - same result on all of them.

BUT if run query select 3.193359374 i get correct float in return: 3.193359374

Here is the list of my numbers (value_in, value_out) than i stored and gotten rounded values:

[(1.529296875, 1.5292969), (-36.2109375, -36.210938), (1.08984375, 1.0898438), (-80.68359375, -80.68359), (70.13671875, 70.13672), (-13.53515625, -13.535156), (-36.03515625, -36.035156), (80.68359375, 80.68359), (-3.33984375, -3.3398438), (1.041015625, 1.0410156), (-4.74609375, -4.7460938), (1.02734375, 1.0273438), (-75.05859375, -75.05859), (-43.41796875, -43.41797), (1.724609375, 1.7246094), (1.072265625, 1.0722656), (87.71484375, 87.71484), (1.595703125, 1.5957031), (53.7890625, 53.789062), (3.791015625, 3.7910156), (1.51953125, 1.5195312), (67.32421875, 67.32422), (-24.2578125, -24.257812), (-45.3515625, -45.351562), (-83.14453125, -83.14453), (4.634765625, 4.6347656), (-36.73828125, -36.73828), (1.576171875, 1.5761719), (-81.9140625, -81.91406), (1.533203125, 1.5332031), (1.70703125, 1.7070312), (1.732421875, 1.7324219), (-24.08203125, -24.082031), (-51.50390625, -51.503906), (1.716796875, 1.7167969), (-49.39453125, -49.39453), (1.04296875, 1.0429688), (1.044921875, 1.0449219), (1.052734375, 1.0527344), (-43.2421875, -43.242188), (67.67578125, 67.67578), (1.00390625, 1.0039062), (-10.72265625, -10.722656), (76.81640625, 76.81641), (-47.28515625, -47.285156), (45.17578125, 45.17578), (1.150390625, 1.1503906), (1.05078125, 1.0507812), (-18.45703125, -18.457031), (-9.31640625, -9.316406), (-85.25390625, -85.25391), (1.599609375, 1.5996094), (-31.81640625, -31.816406), (-72.0703125, -72.07031), (-14.94140625, -14.941406), (-27.0703125, -27.070312), (-42.71484375, -42.714844), (81.73828125, 81.73828), (54.31640625, 54.316406), (4.669921875, 4.669922), (1.15234375, 1.1523438), (-43.9453125, -43.945312), (-68.5546875, -68.55469), (-5.80078125, -5.8007812), (1.14453125, 1.1445312), (1.583984375, 1.5839844), (-7.91015625, -7.9101562), (3.228515625, 3.2285156), (1.525390625, 1.5253906), (-81.03515625, -81.03516), (1.54296875, 1.5429688), (-81.2109375, -81.21094), (4.658203125, 4.658203), (-44.6484375, -44.648438), (1.443359375, 1.4433594), (1.58984375, 1.5898438), (82.08984375, 82.08984), (1.728515625, 1.7285156), (1.03515625, 1.0351562), (-88.76953125, -88.76953), (45.3515625, 45.351562), (4.365234375, 4.3652344), (1.544921875, 1.5449219), (76.46484375, 76.46484), (-6.50390625, -6.5039062), (1.060546875, 1.0605469), (-82.6171875, -82.61719), (-51.6796875, -51.679688), (-78.3984375, -78.39844), (-25.6640625, -25.664062), (-42.5390625, -42.539062), (89.47265625, 89.47266), (4.67578125, 4.6757812), (1.548828125, 1.5488281), (-52.20703125, -52.20703), (4.060546875, 4.060547), (1.033203125, 1.0332031), (59.4140625, 59.414062), (-69.43359375, -69.43359), (-48.69140625, -48.691406), (-8.96484375, -8.964844), (-5.09765625, -5.0976562), (1.73046875, 1.7304688), (89.82421875, 89.82422), (-55.72265625, -55.722656), (-53.0859375, -53.085938), (-10.01953125, -10.019531), (53.61328125, 53.61328), (-71.54296875, -71.54297), (1.001953125, 1.0019531), (53.96484375, 53.964844), (1.55859375, 1.5585938), (72.0703125, 72.07031), (1.056640625, 1.0566406), (-46.0546875, -46.054688), (-25.48828125, -25.488281), (-16.5234375, -16.523438), (1.521484375, 1.5214844), (-34.27734375, -34.277344), (77.16796875, 77.16797), (1.72265625, 1.7226562), (1.41015625, 1.4101562), (-82.44140625, -82.44141), (-16.69921875, -16.699219), (1.541015625, 1.5410156), (1.05859375, 1.0585938), (1.53515625, 1.5351562), (1.064453125, 1.0644531), (1.71484375, 1.7148438), (-46.58203125, -46.58203), (1.07421875, 1.0742188), (66.4453125, 66.44531), (-44.47265625, -44.472656), (-80.5078125, -80.50781), (61.34765625, 61.347656), (67.1484375, 67.14844), (1.52734375, 1.5273438), (3.240234375, 3.2402344), (1.572265625, 1.5722656), (1.048828125, 1.0488281), (-44.82421875, -44.82422), (-16.34765625, -16.347656), (76.11328125, 76.11328), (-34.1015625, -34.101562), (1.60546875, 1.6054688), (63.80859375, 63.808594), (1.517578125, 1.5175781), (1.537109375, 1.5371094), (-76.81640625, -76.81641), (-81.38671875, -81.38672), (-41.66015625, -41.660156), (3.404296875, 3.4042969), (1.591796875, 1.5917969), (-34.98046875, -34.98047), (-24.43359375, -24.433594), (-15.29296875, -15.292969), (-4.39453125, -4.3945312), (-63.80859375, -63.808594), (-12.48046875, -12.480469), (-89.47265625, -89.47266), (4.16015625, 4.1601562), (1.57421875, 1.5742188), (1.556640625, 1.5566406), (1.58203125, 1.5820312), (89.6484375, 89.64844), (-20.7421875, -20.742188), (-6.15234375, -6.1523438), (-39.90234375, -39.902344), (81.38671875, 81.38672), (87.36328125, 87.36328), (-29.1796875, -29.179688), (-66.62109375, -66.62109), (-82.08984375, -82.08984), (4.623046875, 4.623047), (-15.64453125, -15.644531), (3.24609375, 3.2460938), (1.087890625, 1.0878906), (1.69921875, 1.6992188), (-54.66796875, -54.66797), (-51.85546875, -51.85547), (1.705078125, 1.7050781), (3.205078125, 3.2050781), (-79.62890625, -79.62891), (-36.38671875, -36.38672), (54.4921875, 54.492188), (3.22265625, 3.2226562), (1.607421875, 1.6074219), (1.708984375, 1.7089844), (-35.33203125, -35.33203), (-43.06640625, -43.066406), (1.552734375, 1.5527344), (-35.5078125, -35.507812), (88.9453125, 88.94531), (1.029296875, 1.0292969), (-15.99609375, -15.996094), (3.216796875, 3.2167969), (1.720703125, 1.7207031), (1.083984375, 1.0839844), (53.0859375, 53.085938), (-8.61328125, -8.613281), (4.611328125, 4.611328), (-72.24609375, -72.24609), (-87.01171875, -87.01172), (-27.59765625, -27.597656), (1.603515625, 1.6035156), (1.34765625, 1.3476562), (-23.37890625, -23.378906), (-11.77734375, -11.777344), (82.79296875, 82.79297), (1.587890625, 1.5878906), (-83.84765625, -83.84766), (44.6484375, 44.648438), (1.068359375, 1.0683594), (1.373046875, 1.3730469), (-13.88671875, -13.886719), (4.62890625, 4.6289062), (1.513671875, 1.5136719), (1.580078125, 1.5800781), (1.06640625, 1.0664062), (-7.55859375, -7.5585938), (84.90234375, 84.90234), (-32.87109375, -32.871094), (56.6015625, 56.601562), (1.025390625, 1.0253906), (1.59765625, 1.5976562), (-10.37109375, -10.371094), (-45.17578125, -45.17578), (-50.2734375, -50.273438), (4.65234375, 4.6523438), (-17.05078125, -17.050781), (44.82421875, 44.82422), (-74.70703125, -74.70703), (1.55078125, 1.5507812), (4.646484375, 4.6464844), (-13.18359375, -13.183594), (1.701171875, 1.7011719), (-35.68359375, -35.683594), (1.021484375, 1.0214844), (-31.2890625, -31.289062), (-41.1328125, -41.132812), (4.306640625, 4.3066406), (69.43359375, 69.43359), (3.193359375, 3.1933594), (88.76953125, 88.76953), (1.037109375, 1.0371094), (-88.41796875, -88.41797), (-52.3828125, -52.382812), (-89.82421875, -89.82422), (1.712890625, 1.7128906), (-77.6953125, -77.69531), (1.01953125, 1.0195312)]

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Reactions:2
  • Comments:25 (6 by maintainers)

github_iconTop GitHub Comments

2reactions
almibecommented, Sep 1, 2016

Hey I just stumbled on this so sorry if I’m mistaken, but you might be making a very common mistake using Float.MIN_VALUE. It actually isn’t the lowest negative number a Float can hold, it is the smallest positive value a Float can hold. To get the lowest negative number I think you just have to do (Float.MAX_VALUE * -1). I remember running into that before and I think that’s the solution.

1reaction
kivedcommented, Sep 1, 2016

@almibe Nice catch! Given the purpose here (to ensure that the data is stored with the most appropriate type), the test should probably use the absolute value of dou.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Rounding Errors - The Floating-Point Guide
Because floating-point numbers have a limited number of digits, they cannot represent all real numbers accurately: when there are more digits than the...
Read more >
How to round float numbers in javascript? - Stack Overflow
var number = 6.688689; var roundedNumber = Math.round(number * 10) / 10; ... I have very good solution with if toFixed() is not...
Read more >
How to round up a float number in Python? - EasyTweaks.com
Today, we'll learn how to round up float numbers using Python. Solution: We have two methods to handle the rounding float numbers in...
Read more >
4 Examples to Round Floating-Point Numbers in Java up to 2 ...
You can round any floating-point numbers in Java unto n places By using either Math.round(), BigDecimal, or DecimalFormat. I personally prefer to use...
Read more >
Python round() function with EXAMPLES - Guru99
Round () Round() is a built-in function available with python. It will return you a float number that will be rounded to the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found