[Bug]: wrong compute
See original GitHub issueVersion of the software
0.2.5
Operating system
Windows 10
Browser / Native
Native
What happened?
A bug happened! When turn 1GB to Bytes, OB2 will return wrong answer.
Relevant LoliCode if needed
BLOCK:Compute
LABEL:gibtobyte
input = $"1 * 1024 * 1024 * 1024"
=> VAR @computeOutput
ENDBLOCK
Issue Analytics
- State:
- Created 5 months ago
- Comments:5 (1 by maintainers)
Top Results From Across the Web
[BUG] Wrong assignments in compute global/atom with ...
I found a bug in "compute global/atom" in the latest LAMMPS version (lammps-stable_2Aug2023) that caused inconsistent assignments with multiple inputs. LAMMPS ...
Read more >Software bug
A software bug is an error, flaw or fault in the design, development, or operation of computer software that causes it to produce...
Read more >7 types of software bugs and errors
4. Calculation errors · The software is using the wrong algorithm to calculate the value. · The calculation has a data type mismatch....
Read more >What is a bug (computer bug)?
Sometimes referred to as calculation errors, arithmetic bugs are math errors in code that cause it not to function.
Read more >Google Sheets bug causes wrong calculation results
Google Sheets bug causes wrong calculation results. BUG REPORT. Google sheets was showing incorrect calculation results for 2 cells.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I see now what you are talking about, OB2 does in fact do the operation right, but the issue is that it gets wrongly converted back to a string because it gets rounded for some reason. Converting it with dotnet’s ToString() with the “F” formatting indicating to do it as a floating point with no scientific notations gives the proper result.
Not sure what are you talking about because a calculator, a random webpage to convert GB to bytes and my response of adding the “Round to int” block give the exact same answer to that operation, which is 1073741824.
But I noticed that trying to convert 2GB or more will error with
[Executing block Compute] OverflowException: Value is either too large or too small for Type 'Int32'
which is a problem, but a different one.