Feature to change default metrics for memory calculation from RSS to PSS

Problem

jupyter-resource-usage exaggerates usage probably due to copy-on-write. Shifting the methodology from RSS to PSS can help solve the problem. See the below reproducer -

import multiprocessing
import time
import psutil

def foo():
    time.sleep(10000)

parents_original_memory = bytearray(int(1e9)) # 1GB

for i in range(10):
    multiprocessing.Process(target=foo).start()

def get_memory_info(type):
    process_metric_value = lambda process: getattr(process.memory_full_info(), type)
    current_process = psutil.Process()
    all_processes = [current_process] + current_process.children(recursive=True)
    return (
        f"{sum([process_metric_value(process) for process in all_processes]) / 1e9} GB"
    )

print("RSS: ", get_memory_info("rss"))
print("PSS: ", get_memory_info("pss"))

Output is -

RSS: 11.590012928 GB 
PSS: 1.082778624 GB

jupyter-resource-usage reports RSS in JupyterLab, but PSS seems more accurate.

Proposed Solution

• Changing RSS to PSS. But, as per this comment, PSS is a linux-only feature.
• Another solution can be - doing memory calculation via both methods - RSS & PSS (if supported by OS) and let users easily override / change the default metric - through some configuration or by introducing a settings page for the extension.

Additional context

Found similar issue - issues#16

[jtpio]

Just a clarification, I am guessing with the new change, by default the frontend will also use pss and if not available fallback to rss otherwise?

Right, and this would be handled on the server directly similar to https://github.com/ipython/ipykernel/pull/948.

[jtpio]

Thanks @nishikantparmariam for the suggestion 👍

Changing RSS to PSS. But, as per this comment, PSS is a linux-only feature.

Looks like a similar change landed in ipykernel: https://github.com/ipython/ipykernel/pull/948

Maybe it would make sense to return pss when available, and default to rss otherwise (like it is now)?

While keeping rss as the field in the response to avoid breaking other consumers of the API if possible:

https://github.com/jupyter-server/jupyter-resource-usage/blob/559710d106743160180abfefb065512fd1ac137f/jupyter_resource_usage/api.py#L48