Increase speed by parallel execution

We can speed up the process if we do all three steps at the same time - first ffmpeg process creates sound and images (sound should be in some sort of streaming container), python stuff does it’s job frame by frame and feeds those frames to the second ffmpeg. This shouldn’t be too hard to implement with pipes.

[carykh]

That’s true, and I definitely think this would help more for really long videos. It might even work on livestreamed videos! (Although… that doesn’t really make sense since the timeline is cut.) Anyway, I’ll definitely work on piping the processes together when I get some free time again.

(One thing to note is that the python stuff (analyzing the audio) happens much faster than ffmpeg. So as the parallelized code is running, there will essentially be the two ffmpeg processes churning away, and the Python process waiting for 95% of the time.)

[Tinguely]

@carykh

In response to live-streaming:

As an aside to write speed, I think the benefit to a purely in memeory implementation is that you could then stitch in memory instead of offloading to ffmpeg. I’m going to rewrite this in go, I’m pretty sure you could then preform this live with a reasonable buffer instead of having to parse then restitch to view. A buffer would allow the process to run in a way where audio and video is preprocessed a few seconds ahead, since the video doesn’t care about what happened before and after u could apply an algorithm as such:

Define i = 0 Calculate avg audio output for given chunk X (a predetermined Length of audio equal to 1 frame) If X avg audio value < threshold Then Remove audio (doesn’t seem to be needed at 20x)

If i++ % 20 == 0 Then Added frame to video buffer End if

Else Apply same logic as above but keep audio, speed up audio for given frame, if the frame % speed up multiplyier is 0 then add frame to buffer but regardless add the sped up audio to buffer. End else

The advantages of this approach is it doesn’t have to look ahead and the image/audio can be separated into two streams and never has to be encoded (unless being saved but it looks like you have that logic down). Another advantage to this approach is it has no disk write ops which will bottle beck unlike stated above. Hard drives have limited write speed, writing a ton of images is going to take time. I don’t u derstand why there are other comments stating that wouldn’t be the case, any modern cpu is going to be able to rip through images faster than they can be written.