Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Dispatch Router crashes with memory corruption error

See original GitHub issue

Testing with 1000 devices sending telemetry messages with a target throughput over all devices of 200 messages per second, we have noticed that the dispatch router crashed multiple times with the following error:

*** Error in `/sbin/qdrouterd’: malloc(): memory corruption: 0x00007f135c274310 *** ======= Backtrace: ========= /lib64/libc.so.6(+0x791eb)[0x7f1374ee61eb] /lib64/libc.so.6(+0x8431e)[0x7f1374ef131e] /lib64/libc.so.6(+0x84c8e)[0x7f1374ef1c8e] /lib64/libc.so.6(posix_memalign+0x11d)[0x7f1374ef5fbd] /usr/lib/qpid-dispatch/libqpid-dispatch.so(sys_mutex+0x21)[0x7f1375484601] /usr/lib/qpid-dispatch/libqpid-dispatch.so(qd_message+0x116)[0x7f137547d946]

The router has restarted itself and crashed again after a while. The result was that a lot of messages being sent were lost.

Version of EnMasse in the test was 0.15.3.

Version of the router as per the log file is:

2018-01-08 12:14:58.792885 +0000 ROUTER (info) Version: 1.0.0

Complete log file attached.

Issue Analytics

State:
Created 6 years ago
Comments:7 (4 by maintainers)

Top GitHub Comments

1reaction

chapitoscommented, May 24, 2018

@lulf I have not seen this since. So I would say we can close it and open new one if it occurs again.

0reactions

lulfcommented, May 23, 2018

@chapitos This still an issue or can we close this?

Top Results From Across the Web

CSCuv22571 - Memory corruption crash in ... - Cisco Bug

Symptom: After applying IP SLA configuration to prob UDP jitter, we may see router crash with memory corruption. Sample symptoms of crash ......

Routing engineer resolves memory corruption bug in under 10 ...

We had a bug in our legacy C code where a process was attempting to use memory already freed (Use After Free type...

8024s under heavy load could restart because of memory ...

We see a 8024 under heavy load (fw 4.2.1.3) rebooting itsself randomly but very rarely, one time in about 11 months. anyways -...

Investigating memory access crashes - Apple Developer

Memory corruption occurs when a memory location is unexpectedly modified. After this modification, another part of your app may crash when it tries...

Asp.net stress app crash due to memory corruption on Linux

The cited thread does appear to be the source of the seg fault, but it doesn't appear to be related to a bad...