Stream-GlobalTable left join not working as expected
See original GitHub issueDescription
Performing a Stream-GlobalTable left join does not pickup matching keys. Right side values (from table) are always empty/null. The same logic implemented in Java with Streams API works correctly and picks up the matching keys correctly.
How to reproduce
Topic 1 : JoinSample-1 Messages --> Key:Value 100:tableValue-1 200:tableValue-2 300:tableValue-3
Topic 2 : JoinSample-2 Messages --> Key:Value 100:streamValue-4 300:streamValue-3 50:streamValue-8 500:streamValue-2 600:streamValue-1 700:streamValue-5 800:streamValue-6 900:streamValue-7
EXPECTED JoinedEvent Messages --> Key:Value 100:streamValue-4joinedWithtableValue-1 300:streamValue-3joinedWithtableValue-3 50:streamValue-8joinedWith 500:streamValue-2joinedWith 600:streamValue-1joinedWith 700:streamValue-5joinedWith 800:streamValue-6joinedWith 900:streamValue-7joinedWith
ACTUAL JoinedEvent Messages (fails to pickup values for Key 100 and 300 from Table topic) --> Key:Value 100:streamValue-4joinedWith 300:streamValue-3joinedWith 50:streamValue-8joinedWith 500:streamValue-2joinedWith 600:streamValue-1joinedWith 700:streamValue-5joinedWith 800:streamValue-6joinedWith 900:streamValue-7joinedWith
Checklist
Please provide the following information:
- A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file. - code attached
- A code snippet with your topology builder (ex: builder.Stream<string, string>(“topic”).to(“an-another-topic”)😉 - see attached code
- Streamiz.Kafka.Net nuget version. - 1.3.0
- Apache Kafka version. - 6.0.1-ce Confluent Server
- Client configuration. -->
var config = new StreamConfig<StringSerDes, StringSerDes>
{
ApplicationId = $"test-kstreams-{Guid.NewGuid()}",
BootstrapServers = "localhost:29092",
AutoOffsetReset = AutoOffsetReset.Earliest
};
- Operating system. - linux
- Provide logs (with in debug mode (log4net and StreamConfig.Debug) as necessary in configuration). - attached file
- Critical issue - NA Program.txt logs.txt .
Issue Analytics
- State:
- Created a year ago
- Reactions:1
- Comments:7 (4 by maintainers)
Top GitHub Comments
Wonderful. I tested and looks like it is working as expected. Thanks for the quick fix.
Hi @LGouellec ,
Yes, I can confirm that both topics have messages before I run the streams app. Also I made sure that the timestamps for the topic I want to use as table (JoinSample-1 above) are before the topic I want to use as stream (JoinSample-2).