Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Feature to change default metrics for memory calculation from RSS to PSS

See original GitHub issue


jupyter-resource-usage exaggerates usage probably due to copy-on-write. Shifting the methodology from RSS to PSS can help solve the problem. See the below reproducer -

import multiprocessing
import time
import psutil

def foo():

parents_original_memory = bytearray(int(1e9)) # 1GB

for i in range(10):

def get_memory_info(type):
    process_metric_value = lambda process: getattr(process.memory_full_info(), type)
    current_process = psutil.Process()
    all_processes = [current_process] + current_process.children(recursive=True)
    return (
        f"{sum([process_metric_value(process) for process in all_processes]) / 1e9} GB"

print("RSS: ", get_memory_info("rss"))
print("PSS: ", get_memory_info("pss"))

Output is -

RSS: 11.590012928 GB 
PSS: 1.082778624 GB

jupyter-resource-usage reports RSS in JupyterLab, but PSS seems more accurate.

Proposed Solution

  • Changing RSS to PSS. But, as per this comment, PSS is a linux-only feature.
  • Another solution can be - doing memory calculation via both methods - RSS & PSS (if supported by OS) and let users easily override / change the default metric - through some configuration or by introducing a settings page for the extension.

Additional context

Found similar issue - issues#16

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

jtpiocommented, Nov 22, 2022

Just a clarification, I am guessing with the new change, by default the frontend will also use pss and if not available fallback to rss otherwise?

Right, and this would be handled on the server directly similar to

jtpiocommented, Nov 16, 2022

Thanks @nishikantparmariam for the suggestion 👍

Changing RSS to PSS. But, as per this comment, PSS is a linux-only feature.

Looks like a similar change landed in ipykernel:

Maybe it would make sense to return pss when available, and default to rss otherwise (like it is now)?

While keeping rss as the field in the response to avoid breaking other consumers of the API if possible:

Read more comments on GitHub >

github_iconTop Results From Across the Web

linux - How does smem calculate RSS, USS and PSS?
You can simply see the source code. It's written in python. Basically, USS = sum of /proc/<pid>/smaps Private_clean + Private_dirty PSS = sum...
Read more >
Memory Consumption In Linux - DEV Community ‍ ‍
This article explains a reasonable method to measure memory consumption of a process on Linux. · The Linux tools most commonly used are...
Read more >
How can I correctly calculate the sum of memory used by all ...
Subtract from this 9.8 GB reported by arcstat for ZFS ARC Cache. The remaining result is around 5.2 GB of "real" used memory....
Read more >
Process Memory Management in Linux | Baeldung on Linux
In this tutorial, we'll explore four memory measurements used by processes in Linux; VSZ, RSS, USS, and PSS.
Read more >
Memory allocation among processes | Android Developers
Calculating memory footprint · Resident Set Size (RSS): The number of shared and non-shared pages used by the app · Proportional Set Size...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found