question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Incorrect Rounding Applied When resizing

See original GitHub issue

There are certain situation when rounding isn’t applied correctly. Given the snippet below, you can see that rounding of the final two values in the array incorrectly rounds down.

It seems to have something to do with the dtype being used to store the value under the hood. if t_fxp is changed such that the total number of bits <32, then the rounding gets applied correctly.

I have also noticed that this issue is only occuring when the new array is created from an existing FXP array. ie using .resize, or initializing using another FXP array as the initial value. The rounding is correct if initializing from an array of floats. It also looks like the rounding is correct for FXP initialized from a scalar float.

from fxpmath import Fxp, Config
import numpy as np

cfg=Config(dtype_notation="Q",rounding="around")
t_fxp = Fxp(0.0,1,n_int=16,n_frac=15,config=cfg)   # Rounding fails if sign+n_int+n_frac >= 32
t_int = Fxp(0.0,1,n_int=13,n_frac=0,config=cfg)



arr = [-5,0,14.8,7961.625]
fullprec        = Fxp(arr      , like=t_fxp )
rounded_direct  = Fxp(arr      , like=t_int )    # Works
rounded         = Fxp(fullprec , like=t_int )    # Doesn't Work
print(repr(fullprec))
print(repr(rounded_direct))
print(repr(rounded))



scalar_full         = Fxp(7961.625     , like=t_fxp )
scalar_round_direct = Fxp(7961.625     , like=t_int )            #<- Works
scalar_round        = Fxp(scalar_full  , like=t_int )            #<- Also Works
print(repr(scalar_full))
print(repr(scalar_round_direct))
print(repr(scalar_round))

The above will print out

Q17.15([-5.00000000e+00 0.00000000e+00 1.47999878e+01 7.96162500e+03]) Q14.0([-5.000e+00 0.000e+00 1.500e+01 7.962e+03]) Q14.0([-5.000e+00 0.000e+00 1.400e+01 7.961e+03])

Q17.15(7961.625) Q14.0(7962.0) Q14.0(7962.0)

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:8 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
DanChianuccicommented, Apr 21, 2022

Yep! That seems to have fixed the issue

0reactions
francof2acommented, Apr 24, 2022

I close the issue considering it solved in version 0.4.8. Thanks!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Resolve Rounding Errors while Resizing Controls on Winform
Realistically the only way to completely prevent any major rounding errors over the long-term lifetime of a form is to store the original...
Read more >
Resizing rectangle with rounded corner, problem! - Reddit
You've used the function that illustrator gives you to round the corner you have. Which it did to the max, you can't round...
Read more >
Problem resizing rectangle with rounded edge - 10802112
Hi all, How to reduce a rectangle to rounded corners without changing the angles on illustrator You can see taht the form is...
Read more >
Vulkan macOS error VkResult = 1000001003 when resizing ...
This error is happening when the err variable in the example main.cpp is assigned VK_SUBOPTIMAL_KHR . It looks like in this case a...
Read more >
Resizing or Scaling -- IM v6 Examples - ImageMagick
Think of it only applying the resize to images 'greater than' the size given ... Be warned however that the final pixel size...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found