question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

High RAM usage on idle with prometheus 7.0

See original GitHub issue

After upgrading to prometheus-net.AspNetCore “7.0.0” i’m noticing high usage of RAM on my web api projects even on idle but only if you open metrics page at least once. Same behavior with Sample.Web project in this repo. At start RAM usage is not high, but it’s continues to grow with time, so with “7.0.0” version it’s begins to slowly grow after launch from 20 mb up to 70-100mb after 30-60min in my case. Problem disappear if you downgrade to “6.0.0”

Steps to reproduce:

  1. Create project with code below.
  2. Open metrics page “*/metrics” couple of times.

Code:

using Prometheus;

var builder = WebApplication.CreateBuilder(args);

var app = builder.Build();

app.UseMetricServer();
app.UseHttpMetrics();

app.MapGet("/", () => "Hello World!");

app.Run();

csproj:

<Project Sdk="Microsoft.NET.Sdk.Web">

    <PropertyGroup>
        <TargetFramework>net6.0</TargetFramework>
        <Nullable>enable</Nullable>
        <ImplicitUsings>enable</ImplicitUsings>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="prometheus-net.AspNetCore" Version="7.0.0" />
    </ItemGroup>

</Project>

Update So i finded advice from @sandersaares in issue #392 to disable event counter integration and it’s also does the trick and abnormal RAM usage stops.

workaround:

Metrics.SuppressDefaultMetrics(new SuppressDefaultMetricOptions
{
    SuppressEventCounters = true
});

Issue Analytics

  • State:open
  • Created 9 months ago
  • Reactions:1
  • Comments:8 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
sandersaarescommented, Jan 16, 2023

I confirm that I can reproduce this. Interesting find!

It seems to be native memory that is accumulating for some reason. Specifically, the process appears to be accumulating EventCounter event payload values (memory contents closely resemble EventWrittenEventArgs.Payload[n].Values):

I have filed a bug with the .NET runtime, as this seems to be some deep internal wiring: https://github.com/dotnet/runtime/issues/80684

Meanwhile, the workaround you referenced should indeed help you out (assuming you do not need this functionality that is being suppressed). I will continue to explore a permanent solution and will see what the runtime folks say.

1reaction
Jakimarcommented, Feb 6, 2023

After a week of observations i can’t see any problem with version “8.0.0-pre-230127154206-9ec9e9b” both memory and CPU looks totally fine.

Read more comments on GitHub >

github_iconTop Results From Across the Web

RabbitMQ constant memory increase (binary_alloc) in idle ...
After enabling Prometheus today I rechecked memory usage in Grafana for binary_alloc. And it seems like I still have ~15 MB of memory...
Read more >
git - High memory usage for Gitlab CE
When running gitlab on a system with 2Gb ram, this made "idle" ram consumption drop from 1.7Gb to 1.2Gb - So this definitely...
Read more >
High memory usage in idle
3) Type Resource in Search box, open Resource Monitor as Administrator, on Memory tab check the current apportionment of RAM.
Read more >
Is 40% ram usage at idle normal? : r/buildapc
Pretty normal actually. Windows will use memory dynamically, that is to say, if more memory is available, it will use more.
Read more >
How much memory should a PC use when idle?
The amount of memory a PC uses when idle can vary depending on several factors, including the operating system, background processes, and installed ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found