Sending large files over webrtc data channel using Simple peer disconnects the sending peer

I’m trying to send large files of size ~500mb over the data channel to the other peer with of chunk size of 16kb. I’ve noticed that the readstream runs faster and shoots the chunks to the data channel and the writestream on the other side freezes for a while, it almost starts to write when the read stream ends sending the chunks. mean while the sending peer goes offline, but the weird part is the data transfer doesn’t stop until the writestream writes the data. I’m trying to figure out why the sending peer is getting disconnected while data transfer is going on! this.peer = new SimplePeer({ initiator: true, trickle: true, reconnectTimer: 1000, channelConfig: { maxRetransmits: 5, reliable: false, ordered: false, maxPacketLifeTime: 3000 }, iceTransportPolicy: 'all', config: {iceServers: token.iceServers} // Config iceServers comes from twilio token });

[over-engineer]

@Demenus I use a variation of your message queue, thanks for sharing that! 😄

However, for future readers, I think this.webRTCMessageQueue.push(message) in the while loop has to be this.webRTCMessageQueue.unshift(message) so the message gets back at the beginning of the queue. Otherwise, the receiving peer might not receive chunks in order.

I extend simple-peer’s Peer class, so it looks like this:

import SimplePeer from 'simple-peer';

// …

class Peer extends SimplePeer {
  constructor(opts) {
    super(opts);

    this.webRTCPaused = false;
    this.webRTCMessageQueue = [];
  }

  sendMessageQueued() {
    this.webRTCPaused = false;
    
    let message = this.webRTCMessageQueue.shift();

    while (message) {
      if (this._channel.bufferedAmount && this._channel.bufferedAmount > BUFFER_FULL_THRESHOLD) {
        this.webRTCPaused = true;
        this.webRTCMessageQueue.unshift(message);

        const listener = () => {
          this._channel.removeEventListener('bufferedamountlow', listener);
          this.sendMessageQueued();
        };

        this._channel.addEventListener('bufferedamountlow', listener);
        return;
      }

      try {
        super.send(message);
        message = this.webRTCMessageQueue.shift();
      } catch (error) {
        throw new Error(`Error send message, reason: ${error.name} - ${error.message}`);
      }
    }
  }

  send(chunk) {
    this.webRTCMessageQueue.push(chunk);

    if (this.webRTCPaused) {
      return;
    }

    this.sendMessageQueued();
  }
};

// …

export default Peer;

[t-mullen]

My guess is that you are writing the chunks to the peer without throttling. This might be causing your sending peer to run out of memory, or maybe the chunks are overflowing the internal datachannel buffer. Chunk, throttle, write.

If it’s not that, please post how you are reading/chunking/writing to the peer.

You also might want to look at how instant.io+webtorrent implements large file transfer: https://github.com/webtorrent/instant.io/blob/master/client/index.js

FYI: disabling ordered or reliable packets may cause your file to be corrupted.