Incorrect Rounding Applied When resizing
See original GitHub issueThere are certain situation when rounding isn’t applied correctly. Given the snippet below, you can see that rounding of the final two values in the array incorrectly rounds down.
It seems to have something to do with the dtype being used to store the value under the hood. if t_fxp is changed such that the total number of bits <32, then the rounding gets applied correctly.
I have also noticed that this issue is only occuring when the new array is created from an existing FXP array. ie using .resize, or initializing using another FXP array as the initial value. The rounding is correct if initializing from an array of floats. It also looks like the rounding is correct for FXP initialized from a scalar float.
from fxpmath import Fxp, Config
import numpy as np
cfg=Config(dtype_notation="Q",rounding="around")
t_fxp = Fxp(0.0,1,n_int=16,n_frac=15,config=cfg) # Rounding fails if sign+n_int+n_frac >= 32
t_int = Fxp(0.0,1,n_int=13,n_frac=0,config=cfg)
arr = [-5,0,14.8,7961.625]
fullprec = Fxp(arr , like=t_fxp )
rounded_direct = Fxp(arr , like=t_int ) # Works
rounded = Fxp(fullprec , like=t_int ) # Doesn't Work
print(repr(fullprec))
print(repr(rounded_direct))
print(repr(rounded))
scalar_full = Fxp(7961.625 , like=t_fxp )
scalar_round_direct = Fxp(7961.625 , like=t_int ) #<- Works
scalar_round = Fxp(scalar_full , like=t_int ) #<- Also Works
print(repr(scalar_full))
print(repr(scalar_round_direct))
print(repr(scalar_round))
The above will print out
Q17.15([-5.00000000e+00 0.00000000e+00 1.47999878e+01 7.96162500e+03]) Q14.0([-5.000e+00 0.000e+00 1.500e+01 7.962e+03]) Q14.0([-5.000e+00 0.000e+00 1.400e+01 7.961e+03])
Q17.15(7961.625) Q14.0(7962.0) Q14.0(7962.0)
Issue Analytics
- State:
- Created a year ago
- Comments:8 (5 by maintainers)
Top GitHub Comments
Yep! That seems to have fixed the issue
I close the issue considering it solved in version 0.4.8. Thanks!