Kombu disconnects on publish

I’ve posted this elsewhere:-

https://github.com/celery/kombu/issues/463

but I figured it might be considered a separate issue so I created this issue as well.

When I do a lot of high frequency consume and publishes kombu is disconnecting on a publish command with the following error:-

  W_FORCE_CONNECT.format(attr=attr)))
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/nanowire_plugin/__init__.py", line 447, in send_to_next_plugin
    'max_retries':10})
  File "/usr/local/lib/python3.6/site-packages/kombu/messaging.py", line 181, in publish
    exchange_name, declare,
  File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 494, in _ensured
    return fun(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/kombu/messaging.py", line 203, in _publish
    mandatory=mandatory, immediate=immediate,
  File "/usr/local/lib/python3.6/site-packages/amqp/channel.py", line 1745, in basic_publish_confirm
    self.wait(spec.Basic.Ack)
  File "/usr/local/lib/python3.6/site-packages/amqp/abstract_channel.py", line 79, in wait
    self.connection.drain_events(timeout=timeout)
  File "/usr/local/lib/python3.6/site-packages/amqp/connection.py", line 471, in drain_events
    while not self.blocking_read(timeout):
  File "/usr/local/lib/python3.6/site-packages/amqp/connection.py", line 477, in blocking_read
    return self.on_inbound_frame(frame)
  File "/usr/local/lib/python3.6/site-packages/amqp/method_framing.py", line 55, in on_frame
    callback(channel, method_sig, buf, None)
  File "/usr/local/lib/python3.6/site-packages/amqp/connection.py", line 480, in on_inbound_method
    return self.channels[channel_id].dispatch_method(
TypeError: 'NoneType' object is not subscriptable

This is failing to reconnect even after attempting to reconnect 10 times:-

            producer.publish(send_payload, exchange='', routing_key=next_plugin, retry=True, 
                             retry_policy={'interval_start':1,
                                           'interval_step':2,
                                           'interval_max':10,
                                           'max_retries':10})

Also worth noting is that I get the following warning when kombu attempts to reconnect:-

/usr/local/lib/python3.6/site-packages/amqp/connection.py:292: AMQPDeprecationWarning: The .transport attribute on the connection was accessed before
the connection was established.  This is supported for now, but will
be deprecated in amqp 2.2.0.

Since amqp 2.0 you have to explicitly call Connection.connect()
before using the connection.

I assume this warning will disappear when we move to kombu 4.2 but is there any word on the publish disconnect error?

[auvipy]

this is a shame that the issue didn’t get proper attention it needed.! sorry guys! did you found any workaround?

[auvipy]

I experienced the same issue. The problem was in qos prefetch_size. The message queue was too large and prefetch was set to 0, it means that consumer tried to prefetch all queue. The solution would be to reduce qos prefetch_size or if it is not possible, purge message queue and resed it again in smaller batches.

closing the issue based on this.