Rabbitmq becoming inaccessible kills platform instance

Description

RabbitMQ becoming unaccessible kill platform instances running.

|opencti-webError: Channel closed by server: 404 (NOT-FOUND) with message "NOT_FOUND - home node 'rabbit@rabbitmq' of durable queue 'push_2446bb7e-777e-49b7-8f56-001444d403f1' in vhost '/' is down or inaccessible"                                                                                              │
│ opencti-web     at OB.bm.accept (/opt/opencti/build/node_modules/amqplib/lib/channel.js:421:17)                                                                                                                                                                                                                                                                          
│ opencti-web     at XNe.mainAccept [as accept] (/opt/opencti/build/node_modules/amqplib/lib/connection.js:63:33)                                                                                                                                                                                                                                                          
│ opencti-web     at Socket.go (/opt/opencti/build/node_modules/amqplib/lib/connection.js:486:48)                                                                                                                                                                                                                                                                          
│ opencti-web     at Socket.emit (node:events:527:28)                                                                                                                                                                                                                                                                                                                      
│ opencti-web     at emitReadable_ (node:internal/streams/readable:578:12)                                                                                                                                                                                                                                                                                                 
│ opencti-web     at processTicksAndRejections (node:internal/process/task_queues:82:21)

Environment

1. OS (where OpenCTI server runs): 5.3.6 docker image
2. OpenCTI version: 5.3.6
3. OpenCTI client: Platform
4. Other environment details: minikube

Reproducible Steps

Steps to create the smallest reproducible scenario:

1. Run platform
2. Kill or stop or make inaccessible rabbitmq

Expected Output

Platform keep running

Actual Output

Platform instance is killed with unhandled error

Additional information

Screenshots (optional)

[mattseymour]

I will try to replicate this again tomorrow and get the additional logs for you.

On Thu, 16 Jun 2022, 8:26 pm Julien Richard, @.***> wrote:

Can you put some logs just before and after this error? I have done some work to prevent that and not able to reproduce on my side.

My retry testing:

• OCTI 5.3.6 started
• 1 worker started.
• 1 connector started
• Stop the RabbitMQ > ECONNREFUSED errors logs without platform crash
• 1 more connector started > no crash
• Worker restarted > no crash
• Restart RabbitMQ > Everything working well again.

— Reply to this email directly, view it on GitHub https://github.com/OpenCTI-Platform/opencti/issues/2172#issuecomment-1158052316, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKEKOQXATMWBNHUDGNXTI3VPN5V3ANCNFSM5Y6X6M5A . You are receiving this because you authored the thread.Message ID: @.***>

[mattseymour]

So I have been trying to replicate this and I haven’t been able to come up with a successful/ constant way of replicating it. I actually think it could be related to a queue no longer being present but the rabbit instance still running. I will retest this tomorrow, I have been having to juggle a couple of things.

This would match up with the error which is that the queue is no longer present on the rabbitmq instance. I am now sure I was wrong by stating I stopped the rabbitmq instance. Its the fact the queue is no longer present.

Matt Seymour

On Mon, 20 Jun 2022 at 13:27, Samuel Hassine @.***> wrote:

Any news @mattseymour https://github.com/mattseymour? Still not able to reproduce on the latest version on our side.

— Reply to this email directly, view it on GitHub https://github.com/OpenCTI-Platform/opencti/issues/2172#issuecomment-1160386050, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKEKOVSNRVXZIWOYM7LMXLVQBPRRANCNFSM5Y6X6M5A . You are receiving this because you were mentioned.Message ID: @.***>