AckDeadline is not respected

Environment details

• OS: linux
• Node.js version: >12
• npm version: > 6
• @google-cloud/pubsub version: 1.3.0

What are trying to do is to have is a simple fixed-retry using the default ackDeadline.

Steps to reproduce

1. create a subscription with an arbitrary ackDeadline
2. run a subscriber and nack() any message

Message will be redelivered in the next 1-3 seconds regardless of the deadline configuration. Creating the worker with ackDeadline is also seem to be ignored.

example subscriber: topic: test subscription: test, created with deadline: 60

const { PubSub } = require('@google-cloud/pubsub')

const run = async () => {
  const pubsub = new PubSub()

// OR   const subscription = await pubsub.subscription('test', { ackDeadline: 60 })
  const subscription = await pubsub.subscription('test')

  await subscription.get()

  subscription.on('message', msg => {
    const text = msg.data.toString()
    console.log(`${JSON.stringify(new Date())} - ${msg.ackId} - ${msg.data.toString()}`)

    if (text === 'hello') {
      msg.nack()
    } else {
      msg.ack()
    }
  }
}

after publishing a message

➜ node test.js
"2020-01-17T17:21:49.810Z" - projects/gcp-local/subscriptions/test:137 - hello
"2020-01-17T17:21:49.956Z" - projects/gcp-local/subscriptions/test:138 - hello
"2020-01-17T17:21:50.106Z" - projects/gcp-local/subscriptions/test:139 - hello
"2020-01-17T17:21:50.257Z" - projects/gcp-local/subscriptions/test:140 - hello
"2020-01-17T17:21:50.407Z" - projects/gcp-local/subscriptions/test:141 - hello
"2020-01-17T17:21:50.557Z" - projects/gcp-local/subscriptions/test:142 - hello
"2020-01-17T17:21:50.708Z" - projects/gcp-local/subscriptions/test:143 - hello
"2020-01-17T17:21:50.857Z" - projects/gcp-local/subscriptions/test:144 - hello

Our expectation is that the next redelivery would happen only after the deadline (60s).

The behavior of nack() has been changed in #668, but now that hard-codes the deadline to 0 for every message. Looks like that this breaks global ackDeadline settings. Is that a bug or is it an intended side-effect? I found no information on this in the documentation. If its intended, then what is the purpose of the ack deadlines at all?

From the discussion on the PR Its not clear for us why we shouldn’t use the global ackDeadline and nack to remove the messages from the inventory. We cannot go for setTimeout + nack as suggested in the PR because that would cause a massive memory problem. What is the recommended way to implement simple backoff-retry then? Separate topic/subscription or should we use the low level API and implement our own subscriber library or can we remove a message from the inventory somehow without forcing the deadline? Could you please point us in the right direction?

[ettancos]

@feywind thank you very much for checking this. This helps us decide how to move forward. Clarifying the semantics in the documentation sounds great.

that the actual service team doesn’t want us pseudo-implementing ack delays in the client libraries

I’m trying to understand this, let me try to rephrase. In the higher level subscription provided in the library, the ack deadline should only be used as a message processing timeout. Message delay semantics (i.e.: cannot handle the message right now, but it should be redelivered after some time) are not supported by the high level subscription implementation for now. Is that correct?

To clarify, having a method to remove the message from the inventory such as the python client’s drop method would go against the above definition and would result in abuse (I assume from my own problem 😃 ) of it for delay semantics, therefore it is not going to be exposed. Am I understanding this correctly?

How does this relate to the intended semantics of the server side API? Does that mean that using the http/grpc v1 API or the lower level API in the client with such semantics is actually not recommended?

I’m wary of using the older version of the client as we had memory leaks/issues before with certain version, but we will explore that.

[feywind]

I think there isn’t further action here on the nodejs-pubsub side, so I’m going to close this out. I’ve got a few other comments to send to the docs team, so I’ll add this to the list.