Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Slow data channels with big batch download

See original GitHub issue

Hi, I’ve found a CometD issue triggered by the combination of two factors:

  • Slow network infrastructure / connection
  • High volumes of subscribed data in a single batch

and I hope that the CometD protocol can be enhanced in order to take in account such scenario. I’m using the Javascript client and WebSockets.

If the HTTP channel takes more than the configured client-side maxNetworkDelay time to download a batch of data from the server, it can happen that the client code drops the connection with “Message Timeout” error message. (I see that /meta/connect messages will wait more, and precisely (maxNetworkDelay + timeout), but the channel drop can be caused even by other /subscribe messages performed at the beginning of the batch subscription).

What is happening is that the websocket incoming channel will be clogged by a big amount of messages coming from the snapshot of the subscriptions. This causes the responses to /meta/connect and to other /subscription messages to be put in queue, reaching the client potentially after the maxNetworkDelay time, causing the premature channel closure.

I understand that the very reason of such maxNetworkDelay flag is to exactly cover this scenario. However, since I cannot predict the amount of the data that should be downloaded in a batch, I’m forced to put there a high period to cover the worst case scenario (something like 60 seconds in some cases).

This however can hide other issues, in which some message response are really delayed too much when the system load is low (for instance after the batch download is finished).

I was wondering if it would be possible to implement a different logic to let the client to not drop the connection in case of a steady incoming data flow. I was trying to do some experiment in that direction, but I’ve found that in this case, it is the server that will terminate the session: the client will not be able to perform a new /meta/channel request after the server ‘closed’ the previous request.

Is there some additional flag that can be used to cover this scenario, without the need to use huge values of maxNetworkDelay?

Perhaps, I’m just guessing, a good solution could be a sort of ‘heartbeat’ message that the client regularly send to the server, but not related to the /meta/connect traffic, that can inform the server about the client download progress in case of slow channel/big amount of data. When the server receives the periodic hearbeat message (that can contains the of the last message processed), it could adapt the session timeouts to take in account the global end-to-end delay. I think the AckExtensions is not useful in this case, since it only transport information on the /meta/connect message.

What do you think?

Thanks, Luciano

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:7 (5 by maintainers)

github_iconTop GitHub Comments

sbordetcommented, Sep 14, 2020

@lmartorella, after some discussion with colleagues, there may be a better solution - using an extension to dynamically increase the maxNetworkDelay without having to change the CometD internals.

I’ll try to code one in the next days and you can try it out.

sbordetcommented, Oct 23, 2020


Read more comments on GitHub >

github_iconTop Results From Across the Web

How does this happen that when I try to download a large file ...
The small file is being downloaded from that server which is having a slower upload data rate. And vice versa for larger file....
Read more >
Why is my internet so slow? 11 ways to speed up ... - ZDNET
Connectivity drops, bottlenecks, lagged content streaming and downloads, and slow speeds are all common problems with home internet services ...
Read more >
google chrome - Can't download a large amount of data
First, you could compress the file by 7Zip, whose compression is much better than gzip, to reduce the download size.
Read more >
Why Is My Internet So Slow? Try These Tips |
Nothing is more frustrating than slow internet. Find out if you can troubleshoot it or if it's a larger issue with our step-by-step...
Read more >
The hidden cause of slow internet and how to fix it
He termed the phenomenon “bufferbloat.” His conclusion was that critical data packets were trapped in buffers that were excessively large. From ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found