[Issue]: No way to choose Record type for Avro generator lead to ClassCast exception

What happened?

Hello team, I’m trying to produce an avro message using net.coru.kloadgen.sampler.KafkaSchemaSampler. My avro schema contains following field:

{
      "name": "timestamp",
      "type": {
        "type": "long",
        "logicalType": "timestamp-millis"
      }
    },

and there’s an error during serialization:

org.apache.kafka.common.errors.SerializationException: Error serializing Avro message
	at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:154) ~[kloadgen-4.0.9.jar:?]
	at io.confluent.kafka.serializers.KafkaAvroSerializer.serialize(KafkaAvroSerializer.java:61) ~[kloadgen-4.0.9.jar:?]
	at org.apache.kafka.common.serialization.Serializer.serialize(Serializer.java:62) ~[kloadgen-4.0.9.jar:?]
	at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:945) ~[kloadgen-4.0.9.jar:?]
	at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:905) ~[kloadgen-4.0.9.jar:?]
	at net.coru.kloadgen.sampler.KafkaSchemaSampler.runTest(KafkaSchemaSampler.java:164) [kloadgen-4.0.9.jar:?]
	at org.apache.jmeter.protocol.java.sampler.JavaSampler.sample(JavaSampler.java:197) [ApacheJMeter_java.jar:5.4.3]
	at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:638) [ApacheJMeter_core.jar:5.4.3]
	at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:558) [ApacheJMeter_core.jar:5.4.3]
	at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:489) [ApacheJMeter_core.jar:5.4.3]
	at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:256) [ApacheJMeter_core.jar:5.4.3]
	at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.ClassCastException: class java.time.Instant cannot be cast to class java.lang.Number (java.time.Instant and java.lang.Number are in module java.base of loader 'bootstrap') in field timestamp
	at org.apache.avro.generic.GenericDatumWriter.addClassCastMsg(GenericDatumWriter.java:191) ~[kloadgen-4.0.9.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:229) ~[kloadgen-4.0.9.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:210) ~[kloadgen-4.0.9.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:131) ~[kloadgen-4.0.9.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:83) ~[kloadgen-4.0.9.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73) ~[kloadgen-4.0.9.jar:?]
	at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.writeDatum(AbstractKafkaAvroSerializer.java:168) ~[kloadgen-4.0.9.jar:?]
	at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:146) ~[kloadgen-4.0.9.jar:?]	... 11 more
Caused by: java.lang.ClassCastException: class java.time.Instant cannot be cast to class java.lang.Number (java.time.Instant and java.lang.Number are in module java.base of loader 'bootstrap')
	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:160) ~[kloadgen-4.0.9.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:81) ~[kloadgen-4.0.9.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:221) ~[kloadgen-4.0.9.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:210) ~[kloadgen-4.0.9.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:131) ~[kloadgen-4.0.9.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:83) ~[kloadgen-4.0.9.jar:?]
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73) ~[kloadgen-4.0.9.jar:?]
	at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.writeDatum(AbstractKafkaAvroSerializer.java:168) ~[kloadgen-4.0.9.jar:?]
	at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:146) ~[kloadgen-4.0.9.jar:?]	... 11 more

This error happen due to GenericDatumWriter is not suitable for Instant to long conversion. The reason why GenericDatumWriter is used is that generated class by kloadgen is a child of GenericRecord, and KafkaProducer has the following code:

if (value instanceof SpecificRecord) {
    writer = new SpecificDatumWriter(schema);
} else {
    writer = new GenericDatumWriter(schema);
}

Is it possible to add possibility to choose a strategy for avro message generator?

KloadGen Version

latest

Relevant log output

No response

Have you added your JMX Testplan or a replica?

• I have added a JMX testplan

Have you added your Schema or a replica?

• I have added the Schema

[IsaUlloa]

Hi @micr0farad ,

We cannot recommend using one serializer or another, because that depends on what you need in that moment. So everyone should choose the serializer that fits better with their needs.

Nevertheless, we are going to keep this issue open so we can document the characteristics of the serializers we have developed in KLoadGen. This way, we hope this can help people choose the best option for them in the future.

[IsaUlloa]

Hi @micr0farad ,

We will look into it and notify you as soon as possible.