question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Kafka Streams app on native build fails to read json process file

See original GitHub issue

Describe the bug

When running a Kafka Streams native image app second time with an existing state directory, it fails to read the process file with the following exception:

Failed to read json process file: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of org.apache.kafka.streams.processor.internals.StateDirectory$StateDirectoryProcessFile (no Creators, like default constructor, exist): cannot deserialize from Object value (no delegate- or property-based Creator) at [Source: (File); line: 1, column: 2] at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1764) at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:400) at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1209) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1415) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195) at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322) at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3413) at org.apache.kafka.streams.processor.internals.StateDirectory.initializeProcessId(StateDirectory.java:201) at org.apache.kafka.streams.KafkaStreams.<init>(KafkaStreams.java:805) at org.apache.kafka.streams.KafkaStreams.<init>(KafkaStreams.java:782) at org.apache.kafka.streams.KafkaStreams.<init>(KafkaStreams.java:692) at io.quarkus.kafka.streams.runtime.KafkaStreamsProducer.initializeKafkaStreams(KafkaStreamsProducer.java:156) …

When I inspect the org.apache.kafka.streams.processor.internals.StateDirectory, it seams the following fragment causes the problem:

            try {
                final StateDirectoryProcessFile processFileData = mapper.readValue(processFile, StateDirectoryProcessFile.class);
                log.info("Reading UUID from process file: {}", processFileData.processId);
                if (processFileData.processId != null) {
                    return processFileData.processId;
                }
            } catch (final Exception e) {
                log.warn("Failed to read json process file", e);
            }

I think, StateDirectoryProcessFile class being a package protected static inner class causes Jackson to fail when trying to create an instance of it within the native build to deserialize the process file.

I use quarkus-container-image-docker extension to build the native image. Docker file for the native image is the standard Dockerfile.native file which uses quay.io/quarkus/quarkus-micro-image:1.0 base image.

Expected behavior

No response

Actual behavior

No response

How to Reproduce?

No response

Output of uname -a or ver

Linux <hostname> 4.14.252-195.483.amzn2.x86_64 #1 SMP Mon Nov 1 20:58:46 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Output of java -version

java version “11.0.7” 2020-04-14 LTS

GraalVM version (if different from Java)

No local GraalVM installed

Quarkus version or git rev

2.5.1.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)

Additional information

No response

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:10 (9 by maintainers)

github_iconTop GitHub Comments

1reaction
eAybarscommented, May 2, 2022

Sorry guys I forgot about this issue. I will gat right on it this week if thats ok.

1reaction
gsmetcommented, Feb 3, 2022

FWIW, the syntax for a nested class is: org.apache.kafka.streams.processor.internals.StateDirectory$StateDirectoryProcessFile with a $.

Read more comments on GitHub >

github_iconTop Results From Across the Web

running custom built Kafka streams DSL app returns java.lang ...
I was able to extract the entity from the sample JSON you have provided using your JSONExtractor class. Try logging the args input...
Read more >
Streams FAQ | Confluent Documentation
Application fails when running against a secured Kafka cluster?¶. When your application processes data from a secured Kafka cluster, you may run into...
Read more >
Using Apache Kafka Streams - Quarkus
This guide has shown how you can build stream processing applications using Quarkus and the Kafka Streams APIs, both in JVM and native...
Read more >
Error Handling via Dead Letter Queue in Apache Kafka
However, it is Kafka-native. This means you build the complete end-to-end data streaming within a single scalable and reliable ...
Read more >
Documentation - Apache Kafka
The streams API builds on the core primitives Kafka provides: it uses the ... process has started, the source connector should start reading...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found