Redelivery with consumer queue groups

I’m testing JetStream consumer groups and I have a question regarding the redelivery. I’m using nats-server: v2.6.0 and nats.js: v2.2.0

My scenario looks like this:

In this scenario the publisher creates a burst of 10 messages ({ id: 0 }, …, { id: 9 }).

Expected behavior All messages that sub1 rejects to be redelivered.

Actual behavior Some of the messages being redelivered.

Here are consumerOpts of each subscription sub1

ConsumerOptsBuilderImpl {
  stream: '',
  mack: true,
  config: {
    name: '',
    deliver_policy: 'new',
    ack_policy: 'all',
    ack_wait: 30000000000,
    replay_policy: 'instant',
    deliver_group: 'test_unstable_topic',
    durable_name: 'mygroup',
    deliver_subject: 'sub1.test.unstable.topic',
    max_deliver: 5
  },
  qname: 'test_unstable_topic',
  callbackFn: [AsyncFunction (anonymous)]
}

sub 2

ConsumerOptsBuilderImpl {
  stream: '',
  mack: true,
  config: {
    name: '',
    deliver_policy: 'new',
    ack_policy: 'all',
    ack_wait: 30000000000,
    replay_policy: 'instant',
    deliver_group: 'test_unstable_topic',
    durable_name: 'mygroup',
    deliver_subject: 'sub2.test.unstable.topic',
    max_deliver: 5
  },
  qname: 'test_unstable_topic',
  callbackFn: [AsyncFunction (anonymous)]
}

The consumerOpt.callback() is defined here: https://github.com/moleculerjs/moleculer-channels/blob/e61c4596db6d57145e960b785fb85868737326fa/src/adapters/nats.js#L173-L247

and here are the logs:

Logs explanation:

• { id: 0 } - delivered to sub1, got “NACKed” via message.nack() and was NOT redelivered.
• { id: 1 } - delivered to sub2 and logged successfully
• { id: 2 } - delivered to sub2 and logged successfully
• { id: 3 } - delivered to sub2 and logged successfully
• { id: 4 } - delivered to sub2 and logged successfully
• { id: 5 } - delivered to sub2 and logged successfully
• { id: 6 } - delivered to sub2 and logged successfully
• { id: 7 } - delivered to sub1, got “NACKed” via message.nack() and was redelivered again to sub1 and got “NACKed” again. However, after that it was not redelivered. Even with the max_deliver: 5 set.
• { id: 8 } - delivered to sub1, got “NACKed” via message.nack() and was redelivered to sub2 and then logged successfully
• { id: 9 } - delivered to sub1, got “NACKed” via message.nack() and was redelivered to sub2 and then logged successfully

My question is regarding the messages with id=0, id=7, id=8 and id=9. Can you explain why their behavior differs? Is this a config issue?

---

Also, how the load balancing is being done in JetStream? I didn’t find any info about it in the docs. The only reference I found was in Queue Groups which states that one member of the group is chosen randomly to receive the message

However, in the image below the message is being constantly redelivered to the same consumer. It seems that load balancing is not working properly

[AndreMaz]

Did you reset your server the consumer configuration/policy etc won’t change once created in the server.

Ups, sorry.

After resetting the server, things seem to be working fine

Here’s my consumer info:

{
  type: 'io.nats.jetstream.api.v1.consumer_info_response',
  stream_name: 'test_unstable_topic',
  name: 'mygroup',
  created: '2021-09-24T14:43:35.5786762Z',
  config: {
    durable_name: 'mygroup',
    deliver_subject: 'desktop-u6prr0v-2838.sub1.test.unstable.topic',
    deliver_group: 'test_unstable_topic',
    deliver_policy: 'new',
    ack_policy: 'explicit',
    ack_wait: 30000000000,
    max_deliver: -1,
    filter_subject: 'test.unstable.topic',
    replay_policy: 'instant',
    max_ack_pending: 20000
  },
  delivered: {
    consumer_seq: 21,
    stream_seq: 10,
    last_active: '2021-09-24T14:43:37.0620984Z'
  },
  ack_floor: {
    consumer_seq: 21,
    stream_seq: 10,
    last_active: '2021-09-24T14:43:37.0621028Z'
  },
  num_ack_pending: 0,
  num_redelivered: 0,
  num_waiting: 0,
  num_pending: 0,
  cluster: {
    leader: 'NDWYJZQCRJRGBOS7DCC36P2MQRWYRXWRBYODQ4TMNHJ5NGOIY43GV7OK'
  },
  push_bound: true
}

and stream info

{
  type: 'io.nats.jetstream.api.v1.stream_info_response',
  config: {
    name: 'test_unstable_topic',
    subjects: [ 'test.unstable.topic' ],
    retention: 'limits',
    max_consumers: -1,
    max_msgs: -1,
    max_bytes: -1,
    max_age: 0,
    max_msgs_per_subject: -1,
    max_msg_size: -1,
    discard: 'old',
    storage: 'file',
    num_replicas: 1,
    duplicate_window: 120000000000
  },
  created: '2021-09-24T14:43:35.5646832Z',
  state: {
    messages: 10,
    bytes: 570,
    first_seq: 1,
    first_ts: '2021-09-24T14:43:37.0227984Z',
    last_seq: 10,
    last_ts: '2021-09-24T14:43:37.0229304Z',
    consumer_count: 1
  }
}

Thank you for the help 👍

[aricart]

Grab the consumer configuration by asking for it, also list the number of consumers for the stream, there should be one - on your config there you have different delivery subjects, but they are not. The durable is written with the first deliver_to, and when the client grabs the consumer, it uses whatever is already on the server.

const jsm = await nc.jetstreamManager();
const ci = await jsm.consumers.info(stream, "n");