start/stop profiling programmatically while target program is running

Is your feature request related to a problem? Please describe. I’d like to use scalene to profile production workloads. I can’t easily reproduce the production load pattern offline, so I can’t just run scalene myprog offline. I can’t start my job as scalene myprog in production either, because 1) I don’t want to incur the overhead, and 2) I don’t want it to profile from the start of program - I only want to profile specific periods and I want to see the report without stopping my job.

Describe the solution you’d like A way to start and stop profiling, and produce reports programmatically from my program. I don’t mind modifying my program.

Describe alternatives you’ve considered I tried py-spy which allows attaching to a running process, but the results don’t look plausible. I would like to get cross validation from scalene.

Additional context

[guoshimin]

I’m interested in CPU profiling primarily. Even if the overhead is low enough that I can enable it in production, I still don’t want it to profile the entire duration of my job. The load varies with time and I’d like to run profiling for the periods of my choosing.

[emeryberger]

Great! Please report your findings (if successful!) here: https://github.com/plasma-umass/scalene/issues/58