Obtaining the aggregated result when a session window closes when using .suppress()

Hello, I have a strange behaviour when using KStreams to aggregate results and SessionWindows with suppress() method to avoid getting partial aggregations.

I use:

SpringBoot 2.1.5
Spring Cloud Greenwich.SR1
Spring Cloud Stream Germantown.RELEASE
Apache Kafka 2.2.0

I have created a sample project in github, which includes the aggregation process as well as a test to be able to send a message to a topic.

return input
      .groupBy((k, v) -> "explicitKey")
      .windowedBy(SessionWindows.with(window).grace(window))
      .aggregate(
        () -> "",
        (aggKey, newValue, valueAggregate) -> String.join(" ", valueAggregate, newValue),
        (aggKey, leftList, rightList) -> String.join("", leftList, rightList),
        Materialized.<String, String, SessionStore<Bytes, byte[]>>as("InfoStore")
          .withKeySerde(Serdes.String())
          .withValueSerde(Serdes.String()))
      .suppress(Suppressed.untilWindowCloses(unbounded()))
      .toStream()
      .map((key, value) -> pair(key.key() + "--" + key.window().toString(), value));

I want to get the last aggregated result when the window closes, which seems what Suppressed.untilWindowCloses(unbounded()) does. Instead, when the window expires nothing happens, and I have to send another message for the same key to the topic in order to get the aggregated result.

I assume this is not the expected behaviour, as it seems the only way to force the aggregated result to be sent when the window closes, to no avail.

Thanks for your help.

[sabbyanandan]

@lukaszkrawiec: Thanks for taking the time to answer the question!

Hi, @hparkerfly. I hope the answer helps. There’s nothing in our control in SCSt to do anything more to get around the core design premise in Kafka Streams.

If you see any other anomalies in the Spring’s integration with Kafka Streams, please feel free to open a new issue.

[lukaszkrawiec]

One possible option if you can identify the dummy messages would be to handle that in the aggregator code, and simply not add them to the aggregate that you are generating.

Another would be to create a state store on top of a KTable, and query it using an InteractiveQueryService. You can find an example on how to do that here: https://github.com/spring-cloud/spring-cloud-stream-samples/tree/master/kafka-streams-samples/kafka-streams-interactive-query-advanced