progress event support

Hi,

I was working on a transcoding platform and I found your module to wrapping ffmpeg. I’m interested to make improvements that would be useful to everyone involved in this niche.

I’m planning to replace Popen.communicate call to achieve the goals:

• Make ffmpy efficient in memory usage
• Give it progress reporting support

Popen.communicate() is inefficient because of stdout and stderr buffering. Let’s see some cases:

Use case 1:

you want to send transcoded data over network as fast as available. If communicate() is used, you must to accumulate the entire transcoded movie on a memory buffer to send it.

Use case 2:

you want to know the transcoding state (output size, time, bitrate) in real time. If communicate() is used, you can’t do it because this method is consuming the stderr pipe

I have a experimental branch with a on_progress callback implemented here: https://github.com/astroza/ffmpy/tree/progress_support

An usage example is:

def on_progress(state):
        print state.time

ff = ffmpy.FFmpeg(
        global_options="-hide_banner -nostdin -y -stats",
        inputs={local_input_path: None},
        outputs={local_output_path: None},
        on_progress=on_progress
)
ff.run(stderr=subprocess.PIPE)

Now I’m working to make it backwards compatible. The branch break your API

UPDATE

https://github.com/astroza/ffmpy/tree/progress_support_v2 keeps backward compatibility

Usage:

def on_progress(state):
        print state.time

ff = ffmpy.FFmpeg(
        global_options="-hide_banner -nostdin -y -stats",
        inputs={local_input_path: None},
        outputs={local_output_path: None}
)
print(ff.run(on_progress=on_progress)[1])

[astroza]

Hi @Benny739, I have had no response from the repo owner. Nevertheless, you can find an working branch (it’s used on a company) here https://github.com/astroza/ffmpy/tree/progress_support_v2

movie_description = ....

def on_progress(state):
       progress = (float(state.time or state.start_time) / movie_description.duration) * 100
       print(progress)

ff = ffmpy.FFmpeg(
        global_options="-hide_banner -nostdin -y -stats",
        inputs={local_input_path: None},
        outputs={local_output_path: None}
)
print(ff.run(on_progress=on_progress)[1])

state object has all what you need

[Benny739]

I found the bug, it’s caused by piping and not closing stdin and stderr. In the wait function after the while loop I close stdin and stderr now, and the warning is gone.

    def wait(self, on_progress=None, stderr_ring_size=30):
        stderr_ring = []
        is_running = True
        stderr_fileno = self.process.stderr.fileno()
        ff_state = FFstate()
        while is_running:
            latest_update = os.read(stderr_fileno, self.update_size)
            if ff_state.consume(latest_update) and on_progress is not None:
                on_progress(ff_state)
            stderr_ring.append(latest_update.decode())
            if len(stderr_ring) > stderr_ring_size:
                del stderr_ring[0]
            is_running = self.process.poll() is None

        stderr_out = str.join("", stderr_ring)

        self.process.stdin.close()
        self.process.stderr.close()

        if self.process.returncode != 0:
            raise FFRuntimeError(self.cmd, self.process.returncode, stderr_out)

        return stderr_out