question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

High idle CPU usage

See original GitHub issue

Greetings,

First of all, thanks for the great library!

I’ve encountered high idle CPU usage. This is especially noticeable when running multiple microservices (~15 instances) on the single AWS ec2 instance.

  • loki4j version: 1.3.0
  • logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <include resource="org/springframework/boot/logging/logback/defaults.xml"/>
    <include resource="org/springframework/boot/logging/logback/console-appender.xml"/>

    <if condition='isDefined("LOKI_URL")'>
        <then>
            <appender name="LOKI" class="com.github.loki4j.logback.Loki4jAppender">
                <batchTimeoutMs>60000</batchTimeoutMs>
                <http>
                    <url>${LOKI_URL}</url>
                </http>
                <format>
                    <label>
                        <pattern>pod=${HOSTNAME},app=${APP_NAME},level=%level</pattern>
                    </label>
                    <message>
                        <pattern>l=%level c=%logger{20} t=%thread | %msg %ex</pattern>
                    </message>
                    <sortByTime>true</sortByTime>
                </format>
            </appender>

            <root level="INFO">
                <appender-ref ref="CONSOLE"/>
                <appender-ref ref="LOKI"/>
            </root>
        </then>
        <else>
            <root level="INFO">
                <appender-ref ref="CONSOLE"/>
            </root>
        </else>
    </if>

    <logger name="org.springframework.web" level="ERROR"/>
</configuration>

After disabling loki4j and enabling promtail idle load returned to normal

I’m not sure what can cause such behavior. Looking forward to your support

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
nehaevcommented, Dec 10, 2021

Loki4j v1.3.1 was published, according to my measurements idle CPU usage should be lower. Feel free to re-open this issue if the fix didn’t work.

0reactions
nehaevcommented, Dec 6, 2021

Thanks for providing the profiling report on CPU usage! The root cause of this issue is this park timeout hardcoded to 1ms. This timeout defines the frequency that Loki4j checks its internal queues for new events to encode or send. Looks like 1ms cause too much thread context switching on idle.

There’s a tradeoff between encode/send latency and CPU usage, but I guess it should be safe to increase its value to 25ms as a temporary fix. In future versions I’ll consider to make a config property for this.

Read more comments on GitHub >

github_iconTop Results From Across the Web

High CPU usage at idle. Win 10 Pro - Microsoft Community
Whenever I check with MSI Afterburner, it says that the CPU usage sits at about 90 percent.
Read more >
System Idle Process High CPU Usage [Solved] - Driver Easy
The high percent of cpu indicates that a large amount of process power is not being used. If it is at 100, 99...
Read more >
Solved: System Idle Process High CPU usage on Windows 10
The System Idle Process is a system process that tells you the percentage of time your CPU is idle. If you see system...
Read more >
Fix: System Idle process is having a high CPU usage
Fix: System Idle process is having a high CPU usage · Fix-1 Clean Boot your computer- · Fix- 2 Disable Startup processes- ·...
Read more >
Why is my Windows System Idle Process so high?
When the processor is idle, it has a high System Idle Process in the CPU column, often between 70 and 90. If you...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found