[Kafka] Allow Json consume to natively read AVRO Message
See original GitHub issueContext
For my organization project, i needed to produce and consume AVRO messages from the Schema Registry so i develop a custom KakfaClient
to produce the AVRO message.
Concerning the consumption, i detected after a debugging phase that the actual KafkaReceiver
is almost able to treat an incoming AVRO message format thanks to the readJson
method
Here is the actual implementation
ConsumerJsonRecord jsonRecord = new ConsumerJsonRecord(thisRecord.key(), null, valueNode, headersMap);
Here is the code which allow parsing AVRO record
jsonRecords.add(new ConsumerJsonRecord(keyNode, null, valueNode, headersMap));
with keyNode
JsonNode keyNode = objectMapper.readTree(key.toString());
I can do a PR if this issue is accepted
Issue Analytics
- State:
- Created 2 years ago
- Reactions:2
- Comments:6 (1 by maintainers)
Top Results From Across the Web
Consume single json but publish multiple AVRO messages
I recently setup a project and tried kafka stream APIs to publish AVRO. Requirement: Need to parse a complex Jsonobject which might have...
Read more >Optimizing Data Storage Using Compact Binary Formats
Once your data is in Avro format, Kafka Connect provides support to read this Avro data and write it in either Avro or...
Read more >Avro Schema Serializer and Deserializer
Start Confluent Platform using the following command: · Verify registered schema types. · Use the producer to send Avro records in JSON as...
Read more >Reading Avro Streams from Confluent Cloud into Apache Druid
Apache Druid can natively consume data out of a Kafka topic. ... In order to parse Avro messages, you first have to enable...
Read more >Data formats - Apache Druid
Apache Druid can ingest denormalized data in JSON, CSV, or a delimited form such ... use a message header that allows the parser...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Thanks @M3lkior. This looks great. Yes please, please go ahead with the PR and assign the following folks as reviewers please. @sparrowV , @respondsid , @dandalavinod, @ddingel, @santhoshTpixler
Will assign additional reviewers if needed.
Released. Available in v 1.3.28