Normalise Mel filters to constant energy of 1

There is some scaling introduced by the Mel filter bank when calculating the Mel spectrogram. As stated in the documentation setting norm = 1 does give constant energy per filter, but their energy is not 1. In fact, if you sum the filters you don’t get a value of 1, but rather an average of 0.0929 with the following distribution (for n_mel = 128):

This is due to the Mel frequency bandwidth of the triangles used for normalisation, but I’m thinking it might make sense to have the sum of the filter weights equal to 1 when normalised, maybe as an additional option.

[bmcfee]

In my opinion, the proposed implementation 2 would be better,

I think I agree with that, and don’t see a need for implementing proposal 1. (Users who really want that behavior can always scale by sr / n_fft post-hoc.)

Where it gets a little tricky is in dealing with FFT normalization modes. By default, numpy’s FFT is unnormalized, though since 1.10 (released in 2016) you can specify norm='ortho' to scale by sqrt(n_fft) – I think at some point it’s worth considering extending stft (and related) methods to expose this functionality, which did not exist when we first designed the API.

However, this raises a bit of a problem for downstream processing like mel. Slaney’s normalization (our current method) cancels out the factor of n_fft, so that mel spectra computed with different frame lengths (but the same sampling rate) should have comparable scale. This is a nice property to have when you want to experiment with things like frequency over-sampling. Proposals 1 and 2 lose this feature, in favor of propagating frame length scaling through to the output. (It typically comes out in the wash of power_to_db anyway, especially when using an adaptive reference point, but it’s nice to account for all variables here.)

However, if we support ortho mode, we lose this property in mel spectra with the current normalization (and proposal 1). Proposal 2 will have consistent behavior though; I would actually prefer a normalization of the form

util.normalize(melfb, norm=norm, axis=1)

where norm can be any supported normalization (l1, l2, inf, None, etc).

---

and the following names are considered as candidates:

I think it’s helpful here to follow the principle of least surprise. For consistency with the rest of the API, the norm= parameter should conform to the values supported by util.norm(). We can add extra modes, encoded by string values, but we should not change the definition of existing conventions (1, 2, inf, None) or add alternate identifiers for existing conventions; L1 should just be norm=1.

As for how to name the current normalization… maybe 'slaney' is okay after all? Since we haven’t been able to come up with a pithy name that isn’t easily misconstrued, referring to it by attribution rather than description could be a nice compromise.

[onkyo14taro]

@bmcfee Thanks for giving many suggestions!

However, this raises a bit of a problem for downstream processing like mel. Slaney’s normalization (our current method) cancels out the factor of n_fft, so that mel spectra computed with different frame lengths (but the same sampling rate) should have comparable scale. This is a nice property to have when you want to experiment with things like frequency over-sampling. Proposals 1 and 2 lose this feature, in favor of propagating frame length scaling through to the output. (It typically comes out in the wash of power_to_db anyway, especially when using an adaptive reference point, but it’s nice to account for all variables here.) However, if we support ortho mode, we lose this property in mel spectra with the current normalization (and proposal 1).

This perspective was new to me.

I think it’s helpful here to follow the principle of least surprise. For consistency with the rest of the API, the norm= parameter should conform to the values supported by util.norm(). We can add extra modes, encoded by string values, but we should not change the definition of existing conventions (1, 2, inf, None) or add alternate identifiers for existing conventions; L1 should just be norm=1.

This made me understand that following the APIs of other existing functions (like normalize) can reduce user confusion.

I think your proposed norm’s option is best.