Enhancements to handling large files in remote scenarios

In discussion with @derekbekoe, @curib, @connor4312, @aeschli, @alexdima, and @chrisdias a few weeks ago, we discussed enhancements to how VS Code handles large files for remote scenarios.

Some observations from @derekbekoe:

• Looks like VS Code blocks/queues file downloads so if I try to open a large file then a small file afterwards, the small file doesn’t open until the large file has finished?
• Related to the above, it appears it’s not possible to cancel a large file download? Closing the file tab doesn’t appear to stop the download.
• There is a warning for some file types (like The file is not displayed in the editor because it is either binary or uses an unsupported text encoding) but should one be added for opening large files in remote scenarios?

Tentatively assigning @connor4312 and @aeschli as we discussed this in the context of Remote Tunnels, but let me know if I should move to the core repo or update assignment.

[bpasero]

Talked with @joaomoreno , distilled 3 things:

• we do not cancel when you close a tab but we cover that in https://github.com/microsoft/vscode/issues/57585
• we can show a warning when opening large files (maybe configurable) where we have custom defaults based on using vscode-remote scheme or not, I think we do not need a new capability for this (see [1] for example)
• the remote connection seems to be busy dealing exclusively with the response from the remote not allowing other IPC messages, so maybe we need some kind of quality of service in our IPC layer to allow for other IPC messages to get through (I am not an expert in our remote management connection implementation)

[1] Warning in Editor

[alexdima]

Thanks for the clarification! You’re correct, implementing this in a way which can utilise the available bandwidth is quite tricky 🤔 . For example, with a client - server latency of 200ms, if the client reads 64KB chunks at a time, then the maximum theoretical bandwidth is 192KB/s:

• 0ms: the client asks for the first 64KB chunk
• 200ms: the server receives the request and sends the first chunk
• 400ms: the client receives the first 64KB chunk and asks for the next chunk
• 600ms: the server receives the request and sends the second chunk
• 800ms: the client receives the second 64KB chunk and asks for the next chunk
• 1000ms: the server receives the request and sends the third chunk
• 1200ms: the client receives the third 64KB chunk
• …

In order to utilise the available bandwidth, I think the IPC layer needs to either expose the underlying’s socket drain events, write buffer size, etc. or maybe we can support returning a Stream or Promise<Stream> natively in the IPC layer and do the right thing internally to align the writing speed with the TCP/IP socket writing speed, all while allowing other IPC messages to go through. cc @joaomoreno