AvroSerializer/SchemaRegistry works on .NET Core but not .NET Framework 4.5.2

Description

I have following the sample code (https://github.com/confluentinc/confluent-kafka-dotnet/tree/master/examples/AvroSpecific) to build a Kafka Avro Producer with C# code. But it works perfectly on .NET Core project, but failed on .NET Framework 4.5.2. Failed reason is key and value serialization error.

How to reproduce

Build and Run the code provided below on Visual Studio 2015 Target Framework 4.5.2

Checklist

Please provide the following information:

• A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.

public class Program { public static void Main(string[] args) { try { string bootstrapServer = “myBootstrapServer”; string topicName = “myTopicName”; string schemaRegistryUrl = “myUrl”; var saslUsername = “username”; var saslPassword=“password”; var sslCaLocation = “sslCaLocation”

            var config = new ProducerConfig
            {
                BootstrapServers = bootstrapServer,
                SecurityProtocol = SecurityProtocol.SaslSsl,
                SaslMechanism = SaslMechanism.ScramSha512,
                SaslUsername = saslUsername,
                SaslPassword = saslPassword,
                SslCaLocation = sslCaLocation,
            };

            var schemaRegistryConfig = new SchemaRegistryConfig
            {
                SchemaRegistryUrl = schemaRegistryUrl,
                SchemaRegistryBasicAuthCredentialsSource = AuthCredentialsSource.UserInfo,
                SchemaRegistryBasicAuthUserInfo = saslUsername + ":" + saslPassword,
                SchemaRegistryRequestTimeoutMs = 5000,
                SchemaRegistryMaxCachedSchemas = 10
            };

            using (var schemaRegistry = new CachedSchemaRegistryClient(schemaRegistryConfig))
            using (var producer =
                    new ProducerBuilder<string, RepublishRequest>(config)
                        .SetKeySerializer(new AvroSerializer<string>(schemaRegistry))
                        .SetValueSerializer(new AvroSerializer<RepublishRequest>(schemaRegistry))
                        .Build())
            {
                RepublishRequest request = new RepublishRequest { EntityType = EntityType.RMSSKU,
                    EntityId = "53486682",
                    Market = Market.US,
                    SellingChannel = SellingChannel.RACK,
                    ApplicationId = "TestApp"
                };

                byte[] val = SerializerAvro.Serialize(request);
                var task = producer.ProduceAsync(topicName, new Message<string, RepublishRequest> { Key = "test", Value = request });
                task.Wait();

                if (task.IsFaulted) {
                    Console.Write($"error producing message: {task.Exception.Message}");
                } else {
                    Console.Write($"produced to: {task.Result.TopicPartitionOffset}");
                }
                producer.Flush(TimeSpan.FromSeconds(10));
            }
        } catch (Exception e) {
            Console.Write($"EXCEPTION: {e}");
        }
    }

}

RepublishRequest is auto generated Avro class.

• Confluent.Kafka nuget version. 1.0.0

• Apache Kafka version.

• Client configuration.

• Operating system. Windows 10

• Provide logs (with “debug” : “…” as necessary in configuration).

• Provide broker log excerpts.

EXCEPTION: System.AggregateException: One or more errors occurred. —> Confluent.Kafka.ProduceException2[System.String,com.nordstrom.rms.republish.RepublishRequest]: Local: Key serialization error ---> System.Net.Http.HttpRequestException: [https://schema-registry.nonprod.us-west-2.aws.proton.nordstrom.com/] HttpRequestException: An error occurred while sending the request. at Confluent.SchemaRegistry.RestService.<ExecuteOnOneInstanceAsync>d__5.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Confluent.SchemaRegistry.RestService.<RequestAsync>d__61.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Confluent.SchemaRegistry.RestService.<RegisterSchemaAsync>d__14.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Confluent.SchemaRegistry.CachedSchemaRegistryClient.<RegisterSchemaAsync>d__13.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Confluent.SchemaRegistry.Serdes.SpecificSerializerImpl1.<Serialize>d__10.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Confluent.SchemaRegistry.Serdes.AvroSerializer1.<SerializeAsync>d__6.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task) at Confluent.Kafka.Producer2.<ProduceAsync>d__51.MoveNext() --- End of inner exception stack trace --- at Confluent.Kafka.Producer2.<ProduceAsync>d__51.MoveNext() — End of inner exception stack trace — at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Task.Wait() at Confluent.Kafka.Examples.SimpleProducer.Program.Main(String[] args) in C:\MyWork\TestApplication\SimpleKafkaProducer\Program.cs:line 79 —> (Inner Exception #0) Confluent.Kafka.ProduceException2[System.String,com.nordstrom.rms.republish.RepublishRequest]: Local: Key serialization error ---> System.Net.Http.HttpRequestException: [https://schema-registry.nonprod.us-west-2.aws.proton.nordstrom.com/] HttpRequestException: An error occurred while sending the request. at Confluent.SchemaRegistry.RestService.<ExecuteOnOneInstanceAsync>d__5.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Confluent.SchemaRegistry.RestService.<RequestAsync>d__61.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Confluent.SchemaRegistry.RestService.<RegisterSchemaAsync>d__14.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Confluent.SchemaRegistry.CachedSchemaRegistryClient.<RegisterSchemaAsync>d__13.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Confluent.SchemaRegistry.Serdes.SpecificSerializerImpl1.<Serialize>d__10.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Confluent.SchemaRegistry.Serdes.AvroSerializer1.<SerializeAsync>d__6.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task) at Confluent.Kafka.Producer2.<ProduceAsync>d__51.MoveNext() --- End of inner exception stack trace --- at Confluent.Kafka.Producer2.<ProduceAsync>d__51.MoveNext()<— Enter to exit

• Critical issue.

[mhowlett]

marking as enhancement since I think we probably want to surface a different top level exception (that derives from ProduceException) for various common problems with SR integration.

[timarape]

@confluentinc it was actually the certs that had issues in my case, its now resolved. Thanks.