Support the following conventions: per channel, per image and per ensemble

per_channel augmentations: an augmentation that is applied on each audio channel separately per_image augmenations: augmentations that are applied on multichannel audio (e,g. swapping the channels, changing the spatial image) per_ensemble (or per_batch?) augmentations: augmentations that are applied to the whole ensemble of sources. E.g. imagine you want to change the energy of the each image but keep the overall mix energy constant or below a certain value

Ref. a comment from @faroit on slack

In my opinion, per channel and per image has the highest priority

[faroit]

I think by per-ensemble, Fabian still talks about one single example, not a batch. For one example, you might have a set of sources and you’d like to augment them with some kind of constraint (random energy but same mixture normalization in his example). Not sure if capture his idea but that’s what I understand.

yes, that exactly what I meant.

Could you please explain it like I’m five

@iver56 even though five year old can’t code: I make a code example 😉

desired_snr_db = ... # random SNR for this mix, to be augmented
signal_power =  ... # compute snr for one target source (which among the other sources)
for source in sources:
    noise_power = ... # compute snr of this source    
    G = (signal_power / noise_power) * 10**(-desired_snr_db / 10)
    source = torch.sqrt(G) * source

# mix down sources ...

I also agree that per channel and per image are a priority.

agree.

Fabian still talks about one single example, not a batch

yes but I think per_batch should be added as well to the list. My number of request would be random source mixing where the source axes is shuffled so that the mix stems from other samples, thus remove the alignment. Does that make sense?

[iver56]

Transforms now support three modes:

• mode=“per_batch”
• mode=“per_example”
• mode=“per_channel”

I think the “per ensemble” scenario also fits within this framework