Serde with custom aggregater
See original GitHub issueCan any please let me know what is the best way to set the serde for the materialized stores with custom aggregater class Following is my aggregation
groupByKey() .windowedBy(TimeWindows.of(1000).until(1000)) .aggregate(DataAggregate::new, (key, value, aggregate) -> { aggregate.add(1); return aggregate; }, Materialized.as("aggstore"))
DataAggregate is the class that holds the aggregation - I need to save that to aggstore
but the default serde I am using is the
spring.cloud.stream.kafka.streams.binder.configuration.commit.interval.ms=1000 spring.cloud.stream.kafka.streams.binder.configuration.default.key.serde=org.apache.kafka.common.serialization.Serdes$StringSerde spring.cloud.stream.kafka.streams.binder.configuration.default.value.serde=org.apache.kafka.common.serialization.Serdes$StringSerde
exception I am getting is
Exception in thread "eventsin-00aacb65-5224-4b2f-858b-fe98bd134b7c-StreamThread-1" java.lang.ClassCastException: sample.streamprocessor.dataaggregator.DataAggregate cannot be cast to java.lang.String at org.apache.kafka.common.serialization.StringSerializer.serialize(StringSerializer.java:28) at org.apache.kafka.streams.state.StateSerdes.rawValue(StateSerdes.java:178) at org.apache.kafka.streams.state.internals.MeteredWindowStore.put(MeteredWindowStore.java:96) at org.apache.kafka.streams.kstream.internals.KStreamWindowAggregate$KStreamWindowAggregateProcessor.process(KStreamWindowAggregate.java:122)
Issue Analytics
- State:
- Created 5 years ago
- Comments:8 (4 by maintainers)
Top GitHub Comments
@vinu - You cannot directly use the
JsonSerde
class like that. However, you can extend from this in your project and use it in properties. For instance,and then
spring.cloud.stream.kafka.streams.binder.configuration.default.value.serde=FooSerde
@vinu what @olegz and @orchesio said above. You need to pass the proper key and value
Serde
when you materialize the state store. Since you are not passing anySerde
s for that, the defaults are kicking in and running into that exception. You need to use theMaterialized.with...
methods for that. If yourDataAggregate
isjson
friendly, you can use theJsonSerde
provided as part of spring-kafka and pass a reference of this to your aggregation. Hope this helps.