Continuously Rising CPU usage by Kafka streams

This issues is related to another one raised earlier on this package: https://github.com/LGouellec/kafka-streams-dotnet/issues/43

We are making use of kafka-streams-dotnet library in a dotnet core microservices based architecture. We have noticed a peculiar behaviour where our pods running streamer applications always end up consuming 100% of CPU resources allocated to them, sometimes even when they are idling and no processing is happening.

Our question here is whether this is the intended behaviour with the streams implementation, since the stream threads are constantly polling the source topics. If not could you help us with the configurations that we have to use to optimise CPU usage.

The streams topology that we are using is :

Streamiz.kafka.net version being used : 1.1.3

Operating system : Linux (Kubernetes node)

var builder = new StreamBuilder();
            builder.Stream<string, string, StringSerDes, StringSerDes>(_streamOptions.Value.SourceTopic, _timestampExtractor)
                .MapValues((key, value) => _messageProcessor.ProcessMessage(value))
                .Filter((key, value) => CheckThis(key,value))
                .To(_streamOptions.Value.SinkTopic);

[fglaeser]

@ayush113, We are having some cpu problem also in our Openshift cluster with netcore 3.1 apps, (No related with Streamiz), we changed the CG to workstation with good results, maybe this could help you.

[ayush113]

Hi @LGouellec

I will try to create a more detailed dump in an idle scenario, currently facing some problems with debug settings, will share the dump file soon.

Thank you