Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

raw consumer doesn't use channel binding contentType configuration

See original GitHub issue

spring-cloud-stream 1.3.0.BUILD-SNAPSHOT

I’m using the kinesis-produce-consume sample project. I would like to produce messages that do not contain the contentType headers, primarily because Kinesis Analytics does not like the mixed content of string based spring-cloud headers and a JSON payload when defining the schema. (Other notable items include the odd first character and inconsistent spacing/characters between each header name and their quoted value)

I can get the producer to send a JSON only payload by setting the headerMode to raw. I am using the following binding configuration.

spring:
  cloud:
    stream:
      default:
        content-type: application/json
      bindings:
        ordersOut:
          destination: test_stream
          content-type: application/json
          producer:
            headerMode: raw
            partitionKeyExpression: "1" 
        ordersIn:
          destination: test_stream
          content-type: application/json
          consumer:
            headerMode: raw

(I’ve also tried contentType, instead of content-type in the configuration)

But the consumer throws the error

Exception in thread "-kinesis-consumer-1" org.springframework.messaging.converter.MessageConversionException: Could not read JSON: Can not deserialize instance of java.lang.String out of START_OBJECT token
 at [Source: [B@6620923; line: 1, column: 1]; nested exception is com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.lang.String out of START_OBJECT token
 at [Source: [B@6620923; line: 1, column: 1], failedMessage=GenericMessage [payload=byte[126], headers={aws_partitionKey=partitionKey-0, id=21bd52e9-dee3-d6b7-35ff-257a4f5d8370, aws_shard=shardId-000000000000, aws_stream=FoundationStream2, aws_sequenceNumber=49579597709739372335703641938042614070948580590958411778, timestamp=1512690196133}]
	at org.springframework.messaging.converter.MappingJackson2MessageConverter.convertFromInternal(MappingJackson2MessageConverter.java:228)
	at org.springframework.messaging.converter.AbstractMessageConverter.fromMessage(AbstractMessageConverter.java:175)
	at org.springframework.messaging.converter.AbstractMessageConverter.fromMessage(AbstractMessageConverter.java:167)
	at org.springframework.messaging.converter.CompositeMessageConverter.fromMessage(CompositeMessageConverter.java:55)
	at org.springframework.cloud.stream.binding.MessageConverterConfigurer$ContentTypeConvertingInterceptor.preSend(MessageConverterConfigurer.java:254)
	at org.springframework.integration.channel.AbstractMessageChannel$ChannelInterceptorList.preSend(AbstractMessageChannel.java:540)
	at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:417)
	at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:375)
	at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
	at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
	at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
	at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:360)
	at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:271)
	at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:188)
	at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:115)
	at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
	at org.springframework.integration.channel.FixedSubscriberChannel.send(FixedSubscriberChannel.java:70)
	at org.springframework.integration.channel.FixedSubscriberChannel.send(FixedSubscriberChannel.java:64)
	at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
	at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
	at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
	at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:188)
	at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter.access$5100(KinesisMessageDrivenChannelAdapter.java:82)
	at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ShardConsumer.processRecords(KinesisMessageDrivenChannelAdapter.java:912)
	at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ShardConsumer.access$3600(KinesisMessageDrivenChannelAdapter.java:688)
	at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ShardConsumer$2.run(KinesisMessageDrivenChannelAdapter.java:822)
	at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ConsumerInvoker.run(KinesisMessageDrivenChannelAdapter.java:1003)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.lang.String out of START_OBJECT token
 at [Source: [B@6620923; line: 1, column: 1]
	at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:270)
	at com.fasterxml.jackson.databind.DeserializationContext.reportMappingException(DeserializationContext.java:1234)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1122)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1075)
	at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:60)
	at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:11)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3814)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2975)
	at org.springframework.messaging.converter.MappingJackson2MessageConverter.convertFromInternal(MappingJackson2MessageConverter.java:215)
	... 29 more

It appears that in AbstractMessageBinder.ReceivingHandler.handleRequestMessage(...) if there is no content-type header found in the message itself, then it uses a DefaultContentResolver whose defaultMimeType is set to null.

If the contentType header is not present, shouldn’t it resort to using the contentType specified in the channel binding configuration?

Once the error is thrown, it appears that the consuming thread has aborted and no longer listens. Is there a way to recover from this?

Issue Analytics

State:
Created 6 years ago
Comments:6 (4 by maintainers)

Top GitHub Comments

1reaction

dmfreycommented, Jan 5, 2018

@olegz per your request

2018-01-05 16:23:34.973  INFO 37890 --- [           -C-1] o.s.c.s.b.k.KafkaMessageChannelBinder$3  : partitions assigned:[target-events-0]
2018-01-05 16:23:34.996 DEBUG 37890 --- [           -C-1] actMessageChannelBinder$ReceivingHandler : org.springframework.cloud.stream.binder.AbstractMessageChannelBinder$ReceivingHandler@46472571 received message: GenericMessage [payload=byte[156], headers={kafka_offset=0, kafka_timestampType=CREATE_TIME, kafka_receivedMessageKey=null, kafka_receivedPartitionId=0, kafka_receivedTopic=target-events, kafka_receivedTimestamp=1515185270778}]
2018-01-05 16:23:36.002 DEBUG 37890 --- [           -C-1] actMessageChannelBinder$ReceivingHandler : org.springframework.cloud.stream.binder.AbstractMessageChannelBinder$ReceivingHandler@46472571 received message: GenericMessage [payload=byte[156], headers={kafka_offset=0, kafka_timestampType=CREATE_TIME, kafka_receivedMessageKey=null, kafka_receivedPartitionId=0, kafka_receivedTopic=target-events, kafka_receivedTimestamp=1515185270778}]
2018-01-05 16:23:38.006 DEBUG 37890 --- [           -C-1] actMessageChannelBinder$ReceivingHandler : org.springframework.cloud.stream.binder.AbstractMessageChannelBinder$ReceivingHandler@46472571 received message: GenericMessage [payload=byte[156], headers={kafka_offset=0, kafka_timestampType=CREATE_TIME, kafka_receivedMessageKey=null, kafka_receivedPartitionId=0, kafka_receivedTopic=target-events, kafka_receivedTimestamp=1515185270778}]
2018-01-05 16:23:38.006 DEBUG 37890 --- [           -C-1] o.s.c.stream.binder.BinderErrorChannel   : preSend on channel 'target-events.targetEventsReadApi.errors', message: EnhancedErrorMessage [payload=org.springframework.messaging.converter.MessageConversionException: Could not read JSON: Can not deserialize instance of java.lang.String out of START_OBJECT token
 at [Source: [B@37563d7d; line: 1, column: 1]; nested exception is com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.lang.String out of START_OBJECT token
 at [Source: [B@37563d7d; line: 1, column: 1], failedMessage=GenericMessage [payload=byte[156], headers={kafka_offset=0, kafka_timestampType=CREATE_TIME, kafka_receivedMessageKey=null, kafka_receivedPartitionId=0, kafka_receivedTopic=target-events, kafka_receivedTimestamp=1515185270778}], headers={kafka_data=ConsumerRecord(topic = target-events, partition = 0, offset = 0, CreateTime = 1515185270778, serialized key size = -1, serialized value size = 156, headers = RecordHeaders(headers = [], isReadOnly = false), key = null, value = [B@37563d7d), id=77f0e9d7-1fd6-647d-cd45-484942398ac8, timestamp=1515187418006}] for original GenericMessage [payload=byte[156], headers={kafka_offset=0, kafka_timestampType=CREATE_TIME, kafka_receivedMessageKey=null, kafka_receivedPartitionId=0, kafka_receivedTopic=target-events, kafka_receivedTimestamp=1515185270778}]
2018-01-05 16:23:38.009 ERROR 37890 --- [           -C-1] o.s.integration.handler.LoggingHandler   : org.springframework.messaging.converter.MessageConversionException: Could not read JSON: Can not deserialize instance of java.lang.String out of START_OBJECT token
 at [Source: [B@37563d7d; line: 1, column: 1]; nested exception is com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.lang.String out of START_OBJECT token
 at [Source: [B@37563d7d; line: 1, column: 1], failedMessage=GenericMessage [payload=byte[156], headers={kafka_offset=0, kafka_timestampType=CREATE_TIME, kafka_receivedMessageKey=null, kafka_receivedPartitionId=0, kafka_receivedTopic=target-events, kafka_receivedTimestamp=1515185270778}]
	at org.springframework.messaging.converter.MappingJackson2MessageConverter.convertFromInternal(MappingJackson2MessageConverter.java:228)
	at org.springframework.messaging.converter.AbstractMessageConverter.fromMessage(AbstractMessageConverter.java:175)
	at org.springframework.messaging.converter.AbstractMessageConverter.fromMessage(AbstractMessageConverter.java:167)
	at org.springframework.messaging.converter.CompositeMessageConverter.fromMessage(CompositeMessageConverter.java:55)
	at org.springframework.cloud.stream.binding.MessageConverterConfigurer$ContentTypeConvertingInterceptor.preSend(MessageConverterConfigurer.java:254)
	at org.springframework.integration.channel.AbstractMessageChannel$ChannelInterceptorList.preSend(AbstractMessageChannel.java:540)
	at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:417)
	at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:375)
	at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
	at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
	at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
	at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:360)
	at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:271)
	at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:188)
	at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:115)
	at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
	at org.springframework.integration.channel.FixedSubscriberChannel.send(FixedSubscriberChannel.java:70)
	at org.springframework.integration.channel.FixedSubscriberChannel.send(FixedSubscriberChannel.java:64)
	at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:115)
	at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:45)
	at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:105)
	at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:188)
	at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.access$200(KafkaMessageDrivenChannelAdapter.java:63)
	at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:372)
	at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:352)
	at org.springframework.kafka.listener.adapter.RetryingAcknowledgingMessageListenerAdapter$1.doWithRetry(RetryingAcknowledgingMessageListenerAdapter.java:79)
	at org.springframework.kafka.listener.adapter.RetryingAcknowledgingMessageListenerAdapter$1.doWithRetry(RetryingAcknowledgingMessageListenerAdapter.java:73)
	at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287)
	at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:180)
	at org.springframework.kafka.listener.adapter.RetryingAcknowledgingMessageListenerAdapter.onMessage(RetryingAcknowledgingMessageListenerAdapter.java:73)
	at org.springframework.kafka.listener.adapter.RetryingAcknowledgingMessageListenerAdapter.onMessage(RetryingAcknowledgingMessageListenerAdapter.java:39)
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeRecordListener(KafkaMessageListenerContainer.java:925)
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeWithRecords(KafkaMessageListenerContainer.java:909)
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:860)
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:738)
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:622)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Can not deserialize instance of java.lang.String out of START_OBJECT token
 at [Source: [B@37563d7d; line: 1, column: 1]
	at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:270)
	at com.fasterxml.jackson.databind.DeserializationContext.reportMappingException(DeserializationContext.java:1234)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1122)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1075)
	at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:60)
	at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:11)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3814)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2975)
	at org.springframework.messaging.converter.MappingJackson2MessageConverter.convertFromInternal(MappingJackson2MessageConverter.java:215)
	... 38 more

0reactions

olegzcommented, Jan 8, 2018

Also, verified that this bug is not present in 2.0 branch due to the earlier consolidation work done by @viniciusccarvalho and @sobychacko

Top Results From Across the Web

Configuration options can be provided to Spring Cloud Stream ...

If the channel is bound as a consumer, it could be bound to multiple destinations and the destination names can be specified as...

Kombu 5.2.4 documentation - Celery

... consumers will not accept incoming data using the untrusted content types. ... To bind the exchange you call the exchange with the...

Channel Bindings for TLS 1.3 RFC 9266 - IETF Datatracker

Channel Bindings for TLS 1.3 (RFC 9266, July 2022)

LDAP Channel Binding and LDAP Signing on RHEL and AD ...

Impact of Microsoft Security Advisory ADV190023 | LDAP Channel Binding and LDAP ... The channel binding type to use can be configured in ......

Steeltoe Stream Reference

For example, setting the configuration key spring:cloud:stream:bindings:input:destination to raw-sensor-data can be used to configure the channel named ...