Since 2.1 -> failed to resolve class name. Class not found

How to ignore the namespace from the “Operation” class in the deserializer?

How to avoid an infinite loop, if on the Kafka Bus is a class, it can’t deserialize?

Since 2.1 and the trusted packages stuff, I can’t use Kafka in our microservice systems anymore.

I have 2 microservices with independent namespaces. Both are writing/reading a Operation class to/from the kafka bus.

In microservice A there is the Operation class in a.event.Operation. In microservice B there is the Operation class in b.event.Operation.

If it should deserialize the Operation Class from microservice B in microservice A, I get an infinite loop, killing every of my 4 cores with this message:

{
	"@timestamp": "2018-10-23T12:20:22.478+02:00",
	"@version": "1",
	"log_msg": "Error while processing: null",
	"logger_name": "org.springframework.kafka.listener.LoggingErrorHandler",
	"thread_name": "courses-administration-listener-0-C-1",
	"level": "ERROR",
	"level_value": 40000,
	"stack_trace": "org.apache.kafka.common.errors.SerializationException: Error deserializing key/value for partition courses-0 at offset 86. If needed, please seek past the record to continue consumption.\nCaused by: org.springframework.messaging.converter.MessageConversionException: failed to resolve class name. Class not found [b.event.Operation]; nested exception is java.lang.ClassNotFoundException: b.event.Operation\n\tat org.springframework.kafka.support.converter.DefaultJackson2JavaTypeMapper.getClassIdType(DefaultJackson2JavaTypeMapper.java:152)\n\tat org.springframework.kafka.support.converter.DefaultJackson2JavaTypeMapper.toJavaType(DefaultJackson2JavaTypeMapper.java:113)\n\tat org.springframework.kafka.support.serializer.JsonDeserializer.deserialize(JsonDeserializer.java:221)\n\tat org.apache.kafka.clients.consumer.internals.Fetcher.parseRecord(Fetcher.java:923)\n\tat org.apache.kafka.clients.consumer.internals.Fetcher.access$2600(Fetcher.java:93)\n\tat org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.fetchRecords(Fetcher.java:1100)\n\tat org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.access$1200(Fetcher.java:949)\n\tat org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:570)\n\tat org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:531)\n\tat org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1154)\n\tat org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1111)\n\tat org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:700)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.lang.Thread.run(Thread.java:748)\nCaused by: java.lang.ClassNotFoundException: de.predic8.courses.registration.event.Operation\n\tat java.net.URLClassLoader.findClass(URLClassLoader.java:381)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:424)\n\tat sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)\n\tat java.lang.ClassLoader.loadClass(ClassLoader.java:357)\n\tat org.springframework.util.ClassUtils.forName(ClassUtils.java:275)\n\tat org.springframework.kafka.support.converter.DefaultJackson2JavaTypeMapper.getClassIdType(DefaultJackson2JavaTypeMapper.java:148)\n\t... 14 common frames omitted\n",
	"service": "courses-administration-backend"
}

KafkaConfig.class

@EnableKafka
@Configuration
public class KafkaConfig {
    @Bean
    KafkaTemplate<String, Operation> producerFactory(KafkaProperties kafkaProperties) {
        StringSerializer keySerializer = new StringSerializer();

        JsonSerializer<Operation> valueJsonSerializer = new JsonSerializer<>();
        valueJsonSerializer.setAddTypeInfo(false);
        valueJsonSerializer.setUseTypeMapperForKey(false);

        return new KafkaTemplate<>(
                new DefaultKafkaProducerFactory<>(
                        kafkaProperties.buildProducerProperties(),
                        keySerializer,
                        valueJsonSerializer));
    }

    @Bean
    ConsumerFactory<String, Operation> consumerFactory(KafkaProperties kafkaProperties) {
        DefaultJackson2JavaTypeMapper typeMapper = new DefaultJackson2JavaTypeMapper();
        typeMapper.setTypePrecedence(Jackson2JavaTypeMapper.TypePrecedence.INFERRED);
        typeMapper.addTrustedPackages("*");

        JsonDeserializer<Operation> valueDeserializer = new JsonDeserializer<>(Operation.class);
        valueDeserializer.setTypeMapper(typeMapper);
        valueDeserializer.setUseTypeMapperForKey(true);

        return new DefaultKafkaConsumerFactory<>(
                kafkaProperties.buildConsumerProperties(),
                new StringDeserializer(),
                valueDeserializer);
    }

    @Bean
    public ConcurrentKafkaListenerContainerFactory<String, Operation> kafkaListenerContainerFactory(ConsumerFactory<String, Operation> consumerFactory) {
        ConcurrentKafkaListenerContainerFactory<String, Operation> factory = new ConcurrentKafkaListenerContainerFactory<>();

        factory.setConsumerFactory(consumerFactory);

        return factory;
    }

    @Bean
    KafkaTemplate<String, String> kafkaHealthTemplate(KafkaProperties kafkaProperties) {
        return new KafkaTemplate<>(
                new DefaultKafkaProducerFactory<>(
                        kafkaProperties.buildProducerProperties(),
                        new JsonSerializer<>(),
                        new JsonSerializer<>()));
    }
}

Listener:

@KafkaListener(id = "customID",
            topicPartitions =
                    {@TopicPartition(topic = "mytopic",
                            partitionOffsets = @PartitionOffset(partition = "0", initialOffset = "0"))})
    public void listen(ConsumerRecord<String, Operation> cr) {
        try {
            Operation op = mapper.convertValue(cr.value(), Operation.class);
            op.logReceive();

            switch (op.getType()) {
                case "A":
                    handler.handleA(op);
                    break;
                case "B":
                    handler.handleB(op);
                    break;
            }
        } catch (IllegalArgumentException e) {
            log.error(e.getMessage());
        }
    }

[garyrussell]

Kafka 2.2 requires Boot 2.1.

[MartinX3]

Thank you very much for your help! Seems fine now!