Could not parse. Unknown encoding. Invalid content type or spec version
See original GitHub issueHi,
I am trying to parse events published by Kafka. These events are audit events, that are published by Kafka Brokers in case of for example, an authentication attempt.
Here are the details of the event:
Header contains :
content-type : application/cloudevents+json
Value Contains:
{
"data" : {
"serviceName" : "crn:///kafka=JAlahGL_TUmLkqN_24KC2A",
"methodName" : "kafka.Authentication",
"resourceName" : "crn:///kafka=JAlahGL_TUmLkqN_24KC2A",
"authenticationInfo" : {
"principal" : "User:kafka",
"metadata" : {
"mechanism" : "SASL_PLAINTEXT/SCRAM-SHA-256",
"identifier" : "kafka"
}
},
"requestMetadata" : {
"client_address" : "/10.44.168.161"
},
"result" : {
"status" : "SUCCESS",
"message" : ""
}
},
"id" : "84040a3c-e5d4-46ea-b92d-462b020c86f8",
"source" : "crn:///kafka=JAlahGL_TUmLkqN_24KC2A",
"specversion" : "1.0",
"type" : "io.confluent.kafka.server/authentication",
"datacontenttype" : "application/json",
"subject" : "crn:///kafka=JAlahGL_TUmLkqN_24KC2A",
"time" : "2021-06-21T13:48:42.377Z",
"confluentRouting" : {
"route" : "confluent-audit-log-events"
}
}
When I try to deserialize this message in a Kafka Consumer, using CloudEventDeserializer.class
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, CloudEventDeserializer.class);
It throws an exception :
org.apache.kafka.common.errors.SerializationException: Error deserializing key/value for partition audit-producer-topic-1-0 at offset 6. If needed, please seek past the record to continue consumption.
Caused by: io.cloudevents.rw.CloudEventRWException: Could not parse. Unknown encoding. Invalid content type or spec version
at io.cloudevents.rw.CloudEventRWException.newUnknownEncodingException(CloudEventRWException.java:201)
at io.cloudevents.core.message.impl.MessageUtils.parseStructuredOrBinaryMessage(MessageUtils.java:80)
Upon inspection of the MesasgeUtils.java class, it has the following check
public static MessageReader parseStructuredOrBinaryMessage(Supplier<String> contentTypeHeaderReader, Function<EventFormat, MessageReader> structuredMessageFactory, Supplier<String> specVersionHeaderReader, Function<SpecVersion, MessageReader> binaryMessageFactory) throws CloudEventRWException {
String ct = (String)contentTypeHeaderReader.get();
if (ct != null) {
EventFormat format = EventFormatProvider.getInstance().resolveFormat(ct);
if (format != null) {
return (MessageReader)structuredMessageFactory.apply(format);
}
if (ct.startsWith("application/cloudevents")) {
throw CloudEventRWException.newUnknownEncodingException();
}
}
According to documentation, application/cloudevents+json is a valid content type, then why is an exception being thrown for that?
Any explanation around this would be useful…
Thanks in advance !!
Issue Analytics
- State:
- Created 2 years ago
- Comments:9 (4 by maintainers)
Hi, Indeed including the jackson dependency resolves the deserialization issue.
Can you try to debug the
EventFormatProvider
and see if theJsonFormat
is loaded inside it? From the stacktrace, the error seems to be that you don’t have theJsonFormat
loaded properly…