Runtime error: DLLNotFoundException: kernel32

Description

I’m getting the following error when running a sample program on CentOS 7: Unhandled Exception: System.DllNotFoundException: kernel32 at (wrapper managed-to-native) Confluent.Kafka.Impl.LibRdKafka+WindowsNative:LoadLibraryEx (string,intptr,Confluent.Kafka.Impl.LibRdKafka/WindowsNative/LoadLibraryFlags) at Confluent.Kafka.Impl.LibRdKafka.Initialize (System.String userSpecifiedPath) [0x000b6] in <f8eec87f7e1a4d588a368976ab219188>:0 at Confluent.Kafka.Consumer…ctor (System.Collections.Generic.IEnumerable1[T] config) [0x0000c] in <f8eec87f7e1a4d588a368976ab219188>:0 at Confluent.Kafka.Consumer2[TKey,TValue]…ctor (System.Collections.Generic.IEnumerable1[T] config, Confluent.Kafka.Serialization.IDeserializer1[T] keyDeserializer, Confluent.Kafka.Serialization.IDeserializer`1[T] valueDeserializer) [0x0013f] in <f8eec87f7e1a4d588a368976ab219188>:0

This is using the librdkafka.so library I pulled and recompiled on CentOS.

Note that the SimpleProducer and SimpleConsumer apps. work fine when running with “dotnet run”. My app. I can’t run using “dotnet run” because my app. needs to be installed as a Spark process using Mobius, which requires running via: sparkclr-submit.sh.

Any ideas please let me know. Thanks for your time.

[mhowlett]

sorry @ngeor - it’s a little bit down in the queue. e.g. I’m going to get #431 in first. I did take a quick look and saw it’s a couple of hundred lines of changes (though they may be simple, I didn’t think about it) which meant I postponed looking at it further. Since this is a pretty critical part of the code I need to consider it / test it very carefully.

[ngeor]

Hi @mhowlett ,

I have created a PR here: #418

To test it, I built the package on my Windows laptop using my branch. Then, I copied the DLLs on a Linux computer and I used them with the Simple Kafka Producer example (Ubuntu 16.04.3 LTS, mono 4.2.1)

• When using the original DLLs from NuGet, I get the expected kernel32 error
• When using my branch, I get an error that it can’t load the native librdkafka library
• When using my branch with a dllmap file, I am able to run the producer sample

The dllmap file as far as I understand needs to be named Confluent.Kafka.dll.config and it needs this content:

<configuration>
  <dllmap dll="librdkafka" target="absolute path to librdkafka.so" />
</configuration>

The file librdkafka.so is already bundled with the librdkafka NuGet package so that’s fine. Probably the same will work on Mac with the dylib file.

The PR has no changes on the .NET Core side of things, only on the .NET Framework side ( where NET45, NET46 or NET47 conditionals are active). The change essentially constitutes in examining the OS and if it’s not Windows then it doesn’t attempt to call the kernel32 import. If the OS is Windows, everything goes as before. From that point of view, it should be safe to merge the PR, but I don’t know how to write a test that proves that a Windows built binary will work on Linux/Mac.

I’d appreciate any feedback you might have on this!