question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Bug]: wrong compute

See original GitHub issue

Version of the software

0.2.5

Operating system

Windows 10

Browser / Native

Native

What happened?

A bug happened! When turn 1GB to Bytes, OB2 will return wrong answer.

Relevant LoliCode if needed

BLOCK:Compute
LABEL:gibtobyte
  input = $"1 * 1024 * 1024 * 1024"
  => VAR @computeOutput
ENDBLOCK

Issue Analytics

  • State:open
  • Created 5 months ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
DarkScriptedcommented, May 27, 2023

I see now what you are talking about, OB2 does in fact do the operation right, but the issue is that it gets wrongly converted back to a string because it gets rounded for some reason. Converting it with dotnet’s ToString() with the “F” formatting indicating to do it as a floating point with no scientific notations gives the proper result.

String test = @computeOutput.ToString("F"); // will return "1073741824.00"
LOG test
1reaction
DarkScriptedcommented, May 27, 2023

It’s a bug and I guess the problem is that OB2 does not support int64 @DarkScripted

Not sure what are you talking about because a calculator, a random webpage to convert GB to bytes and my response of adding the “Round to int” block give the exact same answer to that operation, which is 1073741824.

But I noticed that trying to convert 2GB or more will error with [Executing block Compute] OverflowException: Value is either too large or too small for Type 'Int32' which is a problem, but a different one.

Read more comments on GitHub >

github_iconTop Results From Across the Web

[BUG] Wrong assignments in compute global/atom with ...
I found a bug in "compute global/atom" in the latest LAMMPS version (lammps-stable_2Aug2023) that caused inconsistent assignments with multiple inputs. LAMMPS ...
Read more >
Software bug
A software bug is an error, flaw or fault in the design, development, or operation of computer software that causes it to produce...
Read more >
7 types of software bugs and errors
4. Calculation errors · The software is using the wrong algorithm to calculate the value. · The calculation has a data type mismatch....
Read more >
What is a bug (computer bug)?
Sometimes referred to as calculation errors, arithmetic bugs are math errors in code that cause it not to function.
Read more >
Google Sheets bug causes wrong calculation results
Google Sheets bug causes wrong calculation results. BUG REPORT. Google sheets was showing incorrect calculation results for 2 cells.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found