Bug(?) when batching wav files that are too short

Hello,

I’m running into issues when using upstream models on audio files that are too short. The model I’m using is audio_albert and the process is killed when the batch includes short waveforms but is processed ok individually.

For example, for a batch with wav with lengths [torch.Size([98399]), torch.Size([311360]), torch.Size([70561]), torch.Size([16801]), torch.Size([5280]), torch.Size([30240]), torch.Size([15040]), torch.Size([1280])], the program is killed without further error messages when I run model(wavs), but I am able to run model(wavs[0]), model(wavs[7]) etc without issue.

I think at least a more informative error message or some sort of warning when input waveforms are too short would help.

Thanks for doing all this, great codebase!

[andi611]

Hi!

Sure, here it is:

import torch
import random
random.seed(0)

model = torch.hub.load("s3prl/s3prl", "audio_albert")
lengths = [3898720, 98399, 311360, 70561, 16801, 5280, 30240, 15040, 1280]
wavs = []

for i in lengths:
    wavs.append(torch.rand(i))

for wav in wavs:
    f = model([wav])
    print(f[0].shape)

feats = model(wavs)
print([x.shape for x in feats])

Hi,

I think the problem is you are doing this on CPU instead of GPU. You can simply add .to('cuda') after model and every input to load them to your GPU. An example of doing extraction on GPU is given here. Let us know if you still encounter this problem after moving to GPU, thanks!

Andy

[trangham283]

Hi @andi611

Moving everything on GPU resulted in a more informative error message (OOM). So the issue with the process being “killed” seems to be that CPU doesn’t have enough memory either, but doesn’t give an informative error message?

Thanks for looking into this.