Confluent.Kafka cannot find librdkafka when running from dotnetcore 2.0 in docker

sadkafka.zip Description

Dotnet core 2 app cannot find librdkafka when running in docker image derived from the microsoft/dotnet base image

How to reproduce

Code attached.

• Ensure you have dotnet core sdk 2 installed
• Extract attached arcive
• dotnet publish sadkafka.csproj (if you want; I already built it)
• docker build --tag sadkafka .

If you were to navigate to /bin/Debug/netcoreapp2.0/publish and exec dotnet sadkafka.dll, everything works fine, however if you run the following

• ‘docker run sadkafka’

You should see the following stacktrace

zopadev@ubuntu:/media/sf_git/kafkaschmafka$ docker run sadkafka
Running...
Unhandled Exception: System.TypeInitializationException: The type initializer for 'Confluent.Kafka.Impl.LibRdKafka' threw an exception. ---> System.DllNotFoundException: Unable to load DLL 'librdkafka': The specified module or one of its dependencies could not be found.
 (Exception from HRESULT: 0x8007007E)
   at Confluent.Kafka.Impl.LibRdKafka.NativeMethods.rd_kafka_version()
   at Confluent.Kafka.Impl.LibRdKafka..cctor()
   --- End of inner exception stack trace ---
   at Confluent.Kafka.Impl.LibRdKafka.conf_new()
   at Confluent.Kafka.Impl.SafeConfigHandle.Create()
   at Confluent.Kafka.Producer..ctor(IEnumerable`1 config, Boolean manualPoll, Boolean disableDeliveryReports)
   at Confluent.Kafka.Producer`2..ctor(IEnumerable`1 config, ISerializer`1 keySerializer, ISerializer`1 valueSerializer, Boolean manualPoll, Boolean disableDeliveryReports)
   at sadkafka.Program.Main(String[] args) in D:\git\kafkaschmafka\Program.cs:line 22

Checklist

Please provide the following information:

• Confluent.Kafka nuget version: 0.11.0
• Apache Kafka version: na
• Client configuration: na
• Operating system: ubuntu, windows (works) vs docker: microsoft/dotnet (based on debian:stretch)
• Provide logs (with “debug” : “…” as necessary in configuration) NA
• Provide broker log excerpts NA
• Critical issue Its blocking us (and we have a support contract with you)

[mhowlett]

Currently working on it. I’m re-packaging librdkafka.redist and making some changes to the dotnet library to make native library loading more flexible. I thought it would be done by now but it’s turned out to be quite difficult.

[treziac]

@spadger Use microsoft/dotnet:2-runtime-jessie or 2-sdk-jessie in your docker file, this will work (for reference, this comes from dependencies installed here: https://github.com/dotnet/dotnet-docker/blob/master/2.0/sdk/jessie/amd64/Dockerfile https://github.com/dotnet/dotnet-docker/blob/master/2.0/sdk/stretch/amd64/Dockerfile )