question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Default precision too low

See original GitHub issue

Can the default Number#precision be changed to a higher value?

Because it’s 5 now, and there is no standard way of setting precision across the forest of Sass environments (many not being Ruby), it causes lots of 1-px off errors, e.g. zurb/foundation#4336 and twbs/bootstrap-sass#409.

libsass also sets precision to 5 by default, same as ruby sass. Perhaps a higher value such as 10 is better as it would mean much less configuration pain for everyone?


ratio rounded error
2/3 66.66667% 0.33333333333E-5
1/6 16.66667% 0.33333333333E-5
3/7 42.85714% -0.28571428571E-5
7/11 63.63636% -0.36363636364E-5
7/31 22.58065% 0.48387096774E-5
p = 5
puts [2/3r, 1/6r, 3/7r, 7/11r, 7/31r].map { |r|
  v = r * 100
  rounded = BigDecimal.new(v, p + Math.log10(v).floor + 1)
  "| #{r} | #{rounded.to_s('F')}% | `#{(rounded - v).to_s('E')}` |" }

Issue Analytics

  • State:closed
  • Created 10 years ago
  • Reactions:6
  • Comments:31 (7 by maintainers)

github_iconTop GitHub Comments

2reactions
glebmcommented, Feb 18, 2014

@lolmaus It is caused by Sass rounding “errors”, please see the issues linked above

2reactions
lolmauscommented, Feb 18, 2014

The unwanted offset might be caused by browser rounding erros. See http://www.palantir.net/blog/responsive-design-s-dirty-little-secret

Read more comments on GitHub >

github_iconTop Results From Across the Web

High Recall - Low Precision for unbalanced dataset
The problem is that I have an unbalanced binary class training set (5:2); which is expected to be proportional to the real class...
Read more >
Clojure - How to set the lower precision - Stack Overflow
I know that to set the precision for numbers in Clojure I should use with-precision function and I know that it works only...
Read more >
Losing My Precision: Tips For Handling Tricky Floating Point ...
Generally, though, Excel will revert to the following floating point notation as default: -2.78E-17. To be sure, that's small, but it's not zero....
Read more >
Change variable precision used - MATLAB digits - MathWorks
Increase Precision of Results. By default, MATLAB ® uses 16 digits of precision. For higher precision, use vpa . The default precision for...
Read more >
Why Floating-Point Numbers May Lose Precision
Because this is a very small number, you should employ user-defined tolerance for calculations involving very large numbers. See also.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found