Set a reasonable default value for max_retries to avoid infinite loops

Hi,

Thank you very much for the fantastic work you do with Celery.

I’m using Celery 4.4.7, and I just spent a lot of time understanding why my server froze when calling my_task.delay(). It turned out that I had a name resolution error, and Celery (or more exactly Kombu) was retying forever to connect to the broker. I first tried to fix the infinite loop with broker_connection_max_retries but later realized that broker_transport_options was the culprit because the default for max_retries is None. BTW, the Celery documentation should be more clear about the difference between broker_connection_max_retries and broker_transport_options['max_retries'].

Retrying forever is a bad idea because it prevents the exception from bubbling up and makes errors very hard to diagnose. Please set a reasonable default value for max_retries to avoid this situation. From my experience, each connection retry takes several seconds, so a small value (let’s say 10) is already a very long wait.

Best regards, Benoit

[bblanchon]

As I implied in my original message, I think 10 is a reasonable default, but it could be lower. For example, I use the value 3 in my project, which means a total of 4 connection attempts.

[thedrow]

I think what Asif means is: What do you consider a reasonable default?