Export high quality video?

Thanks for this great library and to the citizens of the UK for diligently paying their TV License to the BBC.

This closed issue touches on the question of exporting video from VideoContext: https://github.com/bbc/VideoContext/issues/76

This does seem like something there could be significant demand for.

So far I’ve been able to export jerky, low frame rate video using WebMWriter: https://github.com/thenickdude/webm-writer-js

I’ve done it successfully like this:

import VideoContext from "videocontext"
import WebMWriter from "webm-writer"
import { exportLocation } from "../utils/paths";

       const canvas = this.refs.canvas
        const ctx = new VideoContext(canvas);
        const videoNode1 = ctx.video(path.join(__static,
            '/landroid_example-I8JoRYblU.mp4'), 0, 2, { muted: true, loop: true });
        videoNode1.start(0);
        videoNode1.stop(5);
        var videoNode2 = ctx.video(path.join(__static, '/soccert-transparent.webm'), 0, 2, { muted: true, loop: true });
        videoNode2.start(2);
        videoNode2.stop(5);
        var videoNode3 = ctx.video(path.join(__static, '/200-puuWV_57.webm'), 0, 2, { muted: true, loop: true });
        videoNode3.start(3);
        videoNode3.stop(5);
        var combineEffect = ctx.compositor(combineDescription);
        videoNode1.connect(combineEffect);
        videoNode2.connect(combineEffect);
        videoNode3.connect(combineEffect);
        combineEffect.connect(ctx.destination);
        ctx.registerCallback("update", function () {
            console.log("new frame");
            videoWriter.addFrame(canvas);
        });
        const exportPath = exportLocation("testExport.webm")
        console.log(exportPath)
        ctx.registerCallback("ended", function () {
            console.log("Playback ended");
            videoWriter.complete().then(function (webMBlob) {
                 var reader = new FileReader()
                reader.onload = function () {
                    var buffer = new Buffer(reader.result)
                    fs.writeFile(exportPath, buffer, {}, (err, res) => {
                        if (err) {
                            console.error(err)
                            return
                        }
                        console.log('video saved')
                    })
                }
                reader.readAsArrayBuffer(webMBlob)
            });
        });
        var videoWriter = new WebMWriter({
            quality: 0.95,    // WebM image quality from 0.0 (worst) to 1.0 (best)
            fileWriter: null, // FileWriter in order to stream to a file instead of buffering to memory (optional)
            fd: null,         // Node.js file handle to write to instead of buffering to memory (optional)
            // You must supply one of:
            frameDuration: null, // Duration of frames in milliseconds
            frameRate: 24,     // Number of frames per second
        });

        ctx.play();

I’m doing this within an Electron application, so I have access to all the node file system stuff.

So as a proof of concept, this is encouraging. My next challenge is to get a high frame rate, high quality export.

Definitely the export is going to have to be “offline”, or, put another way, it’s not going to be “realtime.”

In the closed issue referenced above, this library is mentioned:

https://github.com/spite/ccapture.js/

It doesn’t work yet as a NPM package, see this issue:

https://github.com/spite/ccapture.js/issues/78

And I’ve been unable to even test it due to this issue:

https://github.com/spite/ccapture.js/issues/87

…But, given access to the Node APIs, I am wondering if this is even the right approach?

There are probably lots of ways to go about this, but I am wondering if anyone here has found the way to make this work reliabily and at full frame rate?

[MysteryPancake]

It may still be possible to use MediaRecorder:

Using MediaRecorder sounds like a great approach. To me there are a few problems that remain unsolved with that way of doing things:

What happens if there is buffering?

This could be overcome with requestFrame. “Applications that need to carefully control the timing of rendering and frame capture can use requestFrame() to directly specify when it’s time to capture a frame.” If requestFrame could be hooked into each canvas update, buffering could be avoided (in theory).

It does not capture audio.

Audio can be captured via MediaStream.addTrack(). It may be possible to render the audio and video separately (perhaps using an OfflineAudioContext), then connect them together via addTrack to keep the timing synchronized.

[kingpalethe]

@Unsigno Thanks. I am wondering if this will work. One of the developers here, @gsouquet, wrote on this issue:

Video Context does not provide any ways to know when a frame has been rendered

… and you seem to be suggesting that you have found a stable way to know when a frame has been rendered… you seem to trying to do so with

 function nodesReady(){
                for (var i in videoContext._sourceNodes) {
                    if (!videoContext._sourceNodes[i]._ready && videoContext._sourceNodes[i].state < 4) return false;
                }
                return true;
            }

I am suspicious that this may not reliable, because if it were this straightforward, surely @gsouquet would have pointed this out.

But I will definitely be trying this soon and will post back here with my findings when I do.