question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Not waiting for all data from socket before parsing message

See original GitHub issue

Steps:

  1. ??? I’m not sure what I did to get into this state (maybe have a bunch of gcm notifications queued?), but I’m consistently in it
  2. Get multiple socket data events per gcm notification

Expected: Able to parse notifications

Actual: index out of range errors from protobuf

I think this is also the cause of #9

When you get into this state, if you add a console.log in socket.on('data' like so: console.log('got data', buffer.length)

Than for the first push notification you’ll see the following:

got data 1
got data 88
got data 105
got data 514
got data 695

For subsequent push notifications you’ll see the following:

got data 1026
got data 189

Note that the first push notification always starts with a buffer of length 1.

When you’re in this state NotificationSchema.decode(buffer) always fails with messages like:

Error: invalid wire type 7 at offset 9
    at BufferReader.Reader.skipType (/Users/islam/Projects/superhuman/desktop-electron/node_modules/protobufjs/src/reader.js:375:19)
    at BufferReader.Reader.skipType (/Users/islam/Projects/superhuman/desktop-electron/node_modules/protobufjs/src/reader.js:366:22)
    at Type.DataMessageStanza$decode [as decode] (eval at Codegen (/Users/islam/Projects/superhuman/desktop-electron/node_modules/@protobufjs/codegen/index.js:50:33), <anonymous>:67:5)
    at Type.decode_setup [as decode] (/Users/islam/Projects/superhuman/desktop-electron/node_modules/protobufjs/src/type.js:502:25)
    at onMessageReceived (/Users/islam/Projects/superhuman/desktop-electron/node_modules/push-receiver/src/client/socket/index.js:125:34)
    at Timeout.setTimeout [as _onTimeout] (/Users/islam/Projects/superhuman/desktop-electron/node_modules/push-receiver/src/client/socket/index.js:105:9)
    at ontimeout (timers.js:475:11)
    at tryOnTimeout (timers.js:310:5)
    at Timer.listOnTimeout (timers.js:270:5)

This is because socket.on('data' is not waiting for all the data to arrive before parsing it. I was able to demonstrate this by making a hack fix:

function listen(socket, NotificationSchema, keys, persistentIds) {
  let buf = null
  let timeout = null

  socket.on('data', (buffer) => {
    if (buf) {
      buf = Buffer.concat([buf, buffer])
    } else {
      buf = buffer
    }

    if (!timeout) {
      timeout = setTimeout(() => {
        onMessageReceived(buf, NotificationSchema, keys, persistentIds)
        buf = null
        timeout = null
      }, 1000)
    }
  });
}

This works but isn’t the right fix since it’s race-condition prone.

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:12 (11 by maintainers)

github_iconTop GitHub Comments

2reactions
ibashcommented, Feb 6, 2018

@MatthieuLemoine / @yurynix FYI I started on this and I should have something usable in the next day or two. I’m copying the implementation from chromium (with a few slight simplifications) so hopefully it’s mostly correct.

Also - @MatthieuLemoine would you be opposed to restructuring this library to use more classes? Ideally each connection can be isolated.

0reactions
MatthieuLemoinecommented, Mar 29, 2018

Fixed in v2.0.0

Read more comments on GitHub >

github_iconTop Results From Across the Web

recv() function not waiting for data - python - Stack Overflow
If recv() is returning an empty string, it means that the connection has been closed. The socket may be closed either by the...
Read more >
Socket.Receive won't wait for the whole string? - MSDN
I am very new to socket programming. I am not receiving all the characters back. Could you please help me to get a...
Read more >
Waiting for Activity on Sockets - IBM
To wait for activity on any sockets managed by a TCP/IP application, use a SOCKET TYPE=SELECT command. To wait for activity on the...
Read more >
Python - How to Receive Full Data with the recv() Socket ...
1. Make the socket non-blocking. By doing this, the socket wont wait if there is no data in recv calls. It will continue...
Read more >
Threads and Sockets in Python | p4-mapreduce
join() , it will wait until the second thread finishes before running any more code. Run the example. Two functions run concurrently: main()...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found