InputNormalization: std can return all nan values

If you have a tensor of all negative values (ex. x = torch.FloatTensor([[-100.0,-100.0,-100.0,-100.0,]]), then torch.std(x, dim=0) will return all nan values.

There is no check or warning when this occurs.

[mravanelli]

I think here the solution is to create a numerically stable normalization, just like other normalization modules such as batch norm or layer norm.

On Fri, 12 Feb 2021 at 13:17, Parcollet Titouan [e-mail] wrote:

Such a feature could be very helpful if you can provide us with any pointer

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/speechbrain/speechbrain/issues/493#issuecomment-778358895, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEA2ZVSHDRJ5KM6RKDGFPLLS6VWC5ANCNFSM4XQALXSQ .

[mravanelli]

I checked the code and, as you can see here, it is already numerically stable: https://github.com/speechbrain/speechbrain/blob/0b074e5a414be79e655ab47ad7809d4618ac2da0/speechbrain/processing/features.py#L1120