Application doesn't finish gracefully after client.disconnect

I expect that after a producer/consumer is disconnected it should destroy all RdKafka objects and let the application exit, however it doesn’t happen.

Code example:

const Kafka = require('node-rdkafka');
const producer = new Kafka.Producer({
    'metadata.broker.list': 'localhost:9092'
});
producer.connect(undefined, () => {
    producer.produce({
        topic: 'test_dc.resource_change',
        message: 'test'
    }, () => {
        producer.disconnect(() => {
            console.log('Disconnected');
        });
    });
});

Versions: kafkaOS X El Capitan Expected behaviour: application sends one message to the topic and shuts down. Actual behaviour: application sends one message to the topic but continues running.

A bit of investigation: gdb shows that we have 8 threads running after disconnect was called:

  Id   Target Id         Frame 
* 1    Thread 0x1203 of process 55925 0x00007fff8f611eca in kevent () from /usr/lib/system/libsystem_kernel.dylib
  2    Thread 0x1303 of process 55925 0x00007fff8f60afae in semaphore_wait_trap () from /usr/lib/system/libsystem_kernel.dylib
  3    Thread 0x1403 of process 55925 0x00007fff8f60afae in semaphore_wait_trap () from /usr/lib/system/libsystem_kernel.dylib
  4    Thread 0x1503 of process 55925 0x00007fff8f60afae in semaphore_wait_trap () from /usr/lib/system/libsystem_kernel.dylib
  5    Thread 0x1603 of process 55925 0x00007fff8f60afae in semaphore_wait_trap () from /usr/lib/system/libsystem_kernel.dylib
  6    Thread 0x1703 of process 55925 0x00007fff8f610db6 in __psynch_cvwait () from /usr/lib/system/libsystem_kernel.dylib
  7    Thread 0x1803 of process 55925 0x00007fff8f61107a in select$DARWIN_EXTSN () from /usr/lib/system/libsystem_kernel.dylib
  8    Thread 0x1903 of process 55925 0x00007fff8f61107a in select$DARWIN_EXTSN () from /usr/lib/system/libsystem_kernel.dylib

Threads 1-5 are normal, so whatever prevents shutdown is in threads 6-8, I suspect it’s on thread 6, here’s the backtrace:

#0  0x00007fff8f610db6 in __psynch_cvwait () from /usr/lib/system/libsystem_kernel.dylib
#1  0x00007fff9cd3f728 in _pthread_cond_wait () from /usr/lib/system/libsystem_pthread.dylib
#2  0x000000010079630b in uv_cond_wait ()
#3  0x000000010078a2ab in worker ()
#4  0x0000000100796000 in uv.thread_start ()
#5  0x00007fff9cd3e99d in _pthread_body () from /usr/lib/system/libsystem_pthread.dylib
#6  0x00007fff9cd3e91a in _pthread_start () from /usr/lib/system/libsystem_pthread.dylib
#7  0x00007fff9cd3c351 in thread_start () from /usr/lib/system/libsystem_pthread.dylib
#8  0x0000000000000000 in ?? ()

Any ideas where could the source of a problem be?

[ottomata]

Been doing some testing. I can reproduce this issue in version 0.3.3. Using master, with #42 merged, my process finishes properly after calling disconnect().

So! Looking good to me!

[Pchelolo]

I’ve created https://github.com/edenhill/librdkafka/issues/775 to understand what does Magnus think about this problem.