Error: librdkafka.so.1: cannot open shared object file: No such file or directory

Environment Information

• OS: Ubuntu 18.4
• Node Version: 9.0.0 and 10.7.0
• NPM Version: 5.5.1
• node-rdkafka version: 2.4.1

Steps to Reproduce

$ cat package.json 
{
  "name": "test-rdkafak",
  "scripts": {
    "test": "node test.js"
  },
  "devDependencies": {
    "node-rdkafka": "^2.3.4"
  }
}

$ cat test.js 
var Transform = require('stream').Transform;

const Kafka = require('node-rdkafka');
var stream = Kafka.KafkaConsumer.createReadStream({
  'metadata.broker.list': 'localhost:9092',
  'group.id': 'librd-test',
  'socket.keepalive.enable': true,
  'enable.auto.commit': false
}, {}, {
  topics: 'test',
  waitInterval: 0,
  objectMode: false
});

stream.on('error', function(err) {
  if (err) console.log(err);
  process.exit(1);
});

stream.pipe(process.stdout);

stream.on('error', function(err) {
  console.log(err);
  process.exit(1);
});

stream.consumer.on('event.error', function(err) {
  console.log(err);
})

It seems to install, but not sure. See install log in comment below.

$ npm install

...


$ npm ls
test-rdkafak@ /home/f483/dev/testkafka
└─┬ node-rdkafka@2.3.4
  ├── bindings@1.3.0
  └── nan@2.10.0

The library it says it can’t find does seem to have been built, is this a configuration issue?

$ find | grep librdkafka.so.1
./node_modules/node-rdkafka/build/deps/librdkafka.so.1
./node_modules/node-rdkafka/deps/librdkafka/src/librdkafka.so.1

$ npm run test

> test-rdkafak@ test /home/f483/dev/testkafka
> node test.js

/home/f483/dev/testkafka/node_modules/bindings/bindings.js:88
        throw e
        ^

Error: librdkafka.so.1: cannot open shared object file: No such file or directory
    at Object.Module._extensions..node (module.js:670:18)
    at Module.load (module.js:560:32)
    at tryModuleLoad (module.js:503:12)
    at Function.Module._load (module.js:495:3)
    at Module.require (module.js:585:17)
    at require (internal/module.js:11:18)
    at bindings (/home/f483/dev/testkafka/node_modules/bindings/bindings.js:81:44)
    at Object.<anonymous> (/home/f483/dev/testkafka/node_modules/node-rdkafka/librdkafka.js:10:32)
    at Module._compile (module.js:641:30)
    at Object.Module._extensions..js (module.js:652:10)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! test-rdkafak@ test: `node test.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the test-rdkafak@ test script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/f483/.npm/_logs/2018-08-15T05_36_39_154Z-debug.log

[mike-coolfront]

I’ve not had time to assemble a proper pull request to get my potential fix reviewed. If anyone would be interested in trying out my adjustment here is a link to the commit on my fork. potential fix for issue

The change effects the build configuration used by node-rdkafaka when building librdkafka, so you’ll have to work through rebuilding node-rdkafka and deploying the adjusted version to use it.

[trevorscott]

@mike-coolfront I tested out your fix and it solves this issue when node-rdkafka is deployed on Heroku.