ASB: client just stops receiving (node-amqp10 version 3.1.3)

On our production machine 3 times now, the receiver spontaneously stops working. New messages are delivered to the queue, but we never receive them from the ASB (receiver.on(‘message’… is not called anymore). No error, no nothing.

The first time it happened was after about a week uptime. After that it took 4 weeks before it stopped working and yesterday after only a couple of hours it happened again.

It can’t be anything in our own code, because the receiver looks like this (simplified):

receiver.on('message', function(message) {
    let link = this;
    log(`start receive ${message.properties.messageId}`);
    handleMessage(message);
    link.accept(message);
    log(`message ${message.properties.messageId} accepted`);
}

The log shows:

start receive xxxxx
message xxxxx accepted

And then when (even if it is only a minute later, so no timeout can cause the problem) another message is delivered to the queue it just stays there and nothing happens.

The big problem is that there is no error, so i cannot take action (in code or manually). I just have to keep an eye on the ASB queues and when it stops working i have to restart the application which solves the problem (because the connection is setup again).

Anyone else had this kind of problem?

Of course i could turn on DEBUG logging, but since it is not predictable when the problem occurs (it can take 4 weeks) and we process thousands of messages a day (4 topics for sending and 5 queues for receiving) in our application that would degrade performance i think and log files will be enormous.

I think cause of the problem must be in the ASB (where the connection just gets lost), because when the problem occurs we don’t get the normal timeout after 10 minutes of inactivity (the connection would be setup again and the problem would be temporary (still not nice, but not as bad as it is now)).

I’ll do some more research on this, but at the moment i don’t have an idea how to handle this problem.

[abhijeetahuja]

any updates on this? I am facing the exact same issue.

[amarzavery]

@njLT Some fixes were made in the 3.6.0 version of node-amqp10. Try using that. I still believe this looks like a service side issue since you were getting this error while using the http based azure-sb library and this library as well.