Error "no OPENSSL_AppLink" when trying to configure SSL in .Net producer

Description

Following some more poking, this appears to be a regression with 1.8.0. I’ve gone to Confluent.Kafka 1.6.3 and 1.7.0 and the same code works without changes and I can push messages. As soon as I push to 1.8.0, I get the “no OPENSSL_AppLink” error, which remains through 1.8.1 and 1.8.2. Original report follows:

I am trying to write a piece of C# that can authenticate with a client certificate to a test kafka instance. On running the code, I get the following error when calling ProducerBuilder(), and the code immediately terminates (doesn’t even throw an exception):

OPENSSL_Uplink(00007FF9C9C78D30,08): no OPENSSL_Applink

No errors or messages appear in the Kafka server logs.

If I comment out the lines that set the certificate (i.e., remove the config for SslKeystoreLocation and both passwords), then the error doesn’t occur (but I don’t authenticate either as the Kafka server is configured for mutual authentication), and I do see an “Failed authentication with /10.168.84.10 (SSL handshake failed)” error message in the Kafka logs, so the server is reachable.

Client machine is Windows 10 20H2, C# code is built against .Net Core 3.1. Confluent.Kafka version 1.8.2 Server machine is Linux (Ubuntu 21.04) running Kafka 2.13

C# code: Program.txt

How to reproduce

Execute the code in the attachment (having configured BootstrapServers and a suitable PKCS#12 certificate and password) against a Kafka configured for TLS authentication.

Checklist

Please provide the following information:

• A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.
• Confluent.Kafka nuget version.
• Apache Kafka version.
• Client configuration.
• Operating system.
• Provide logs (with “debug” : “…” as necessary in configuration).
• Provide broker log excerpts. [Reporter note – There are no log entries created server-side, so no log included]

[EnocAnanda]

Now I get a different error. I might not have a good way to test without taking too much time away from my work tasks. The original issue only seems to affect consumers and not when just publishers were used.

Exception has occurred: CLR/System.InvalidOperationException Exception thrown: 'System.InvalidOperationException' in Confluent.Kafka.dll: 'Error reading ssl.keystore.location PKCS#12 file: [snipped]/kafka/tls/client/kafka.client.keystore.jks: error:0D06B08E:asn1 encoding routines:asn1_d2i_read_bio:not enough data' at Confluent.Kafka.Impl.SafeKafkaHandle.Create(RdKafkaType type, IntPtr config, IClient owner) at Confluent.Kafka.Consumer2…ctor(ConsumerBuilder2 builder) at Confluent.Kafka.ConsumerBuilder2.Build()`

I also see the same error when I attempt to use the pre-release RC’s for v1.9.x, whether I include the dependent library as an external package or not. At this point with the C# .NET Client, we have to use v1.7.0 of the Confluent NuGet packages. Nothing above that level works for us. Yes, we are using the keystore approach and we have to for mTLS support.

[mhowlett]

https://github.com/mhowlett/confluent-kafka-dotnet/tree/security/examples/Security