InMemoryWindowStore System.NullReferenceException

Description

When I try to reprocess the messages from earliest offset of a topic that is populating an InMemoryWindowStore, while restoring state and trying to skip record for expired segment, the library crashes on a NullReferenceException. In the class InMemoryWindowStore.cs method Put() line 351, the context.Timestamp variable is NULL. Code : if (windowStartTimestamp <= observedStreamTime - retention.TotalMilliseconds) { expiredRecordSensor.Record(1.0, context.Timestamp); logger.LogWarning("Skipping record for expired segment"); }

How to reproduce

1. Setup a source topic called topic and produce messages to the topic with historical timestamps. Meaning some messages should have timestamp in Feb 2023, some in March 2023, some in April 2023 and finally some messages with current timestamp.
2. Setup a simple windowed aggregation and write the results of the aggregation to an InMemoryWindows store.
3. Setup an application Id.
4. Run the app and let it process to the latest offset.
5. Reset the offset to earliest for the application Id/consumer group.
6. Restart the application to process messages. Program.txt

Checklist

Please provide the following information:

• [ DONE] A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.
• [DONE-Inside attached program] A code snippet with your topology builder (ex: builder.Stream<string, string>(“topic”).to(“an-another-topic”)😉
• [1.4.2] Streamiz.Kafka.Net nuget version.
• Apache Kafka version.
• [ DONE-Inside attached program] Client configuration.
• [Linux ] Operating system.
• [ logs.txt ] Provide logs (with in debug mode (log4net and StreamConfig.Debug) as necessary in configuration).
• [Yes] Critical issue.

[mayur-rao]

Hi @LGouellec

I did a restart of my application and after that no longer running into the thread exception. So please close this issue. I will report back if see an issue again.

Again, I thank you for your quick response and time spent.

[LGouellec]

@mayur-rao ,

Can you share the GlobalTable topology please ?