AudioSegment.overlay() slow when called many times

Hello!

I’m currently working on a tool that renders Note Block Studio songs (.nbs) to audio files. This format is similar to MIDI in that it contains what notes should be played at a given moment, each one with a certain pitch, volume and panning.

Thanks to Pydub, I got this to work flawlessly in less than two days. However, the songs made in this program may easily reach 20,000+ notes, which means calling AudioSegment.overlay() thousands of times. Of course, this slows down everything.

A song with 20k notes takes about 2.5 hours to complete, while a black-MIDI-type song with 150k notes took nearly 14 hours. When removing the overlay() call, however, everything is done within seconds (even with all the pitch/gain/speed changes).

I know the aim of Pydub is not to be highly efficient, but intuitive. What I’d like to know is if there’s any way around the overlay() method that still allows me to keep the other modifications done with Pydub. More specifically:

• When inspecting the source code, I noticed this method is implemented in pure Python. Could it be made more efficient by doing lower-level calls, such as calling ffmpeg to overlay the audio? (I’d be happy to give this a go if it turns out to be possible.)
• Should I use AudioSegment.get_array_of_samples() on each sound and use something else to overlay them? Or…
• Is what I’m trying to accomplish too advanced for Pydub, and I should use a lower-level package instead (e.g. pysox)?

Thanks!

[Peda1996]

Thanks a lot 😄.

[Bentroen]

@Peda1996 Yeah, here it is! https://gist.github.com/Bentroen/4df9b8d5d052f9d14bc1a8531fe49994

Before calling to_audio_segment(), make sure all your sounds have the same sample rate by sync()ing them previously if necessary, otherwise, a MemoryError may occur as it tries to find a common sample rate. 😃