question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[BUG] Progress bar data is not piped when a script using rich is run in a subprocess

See original GitHub issue

Describe the bug

Progress bar updates are not piped via stdout when running any script that uses rich in a subprocess. I specifically noticed this when trying to parse the progress bar of a pip install (pip uses rich for its progress bar), when I noticed that it only pipes the data of the final (finished) bar, and none of the updates.

This stack overflow post I found describes the issue in slightly more detail and provides a minimal repro. They mention using python’s Popen, but the same thing happens in nodejs’s child_process.spawn function.

Here’s an example of what it outputs:

Collecting onnx\r\n
  Downloading onnx-1.11.0-cp39-cp39-win_amd64.whl (11.2 MB)\r\n
---------------------------------------- 11.2/11.2 MB 6.7 MB/s eta 0:00:00\r\n

And what I would expect it to output:

Collecting onnx\r\n
  Downloading onnx-1.11.0-cp39-cp39-win_amd64.whl (11.2 MB)\r\n
     -                                        1.0/11.2 MB 6.7 MB/s eta 0:01:00\r\n
     ----                                     2.2/11.2 MB 6.7 MB/s eta 0:00:50\r\n
     ------------                             6.8/11.2 MB 6.7 MB/s eta 0:00:30\r\n
     ---------------------------              8.2/11.2 MB 6.7 MB/s eta 0:00:20\r\n
     ------------------------------------     10.0/11.2 MB 6.7 MB/s eta 0:00:10\r\n
     ---------------------------------------- 11.2/11.2 MB 6.7 MB/s eta 0:00:00\r\n

Platform Windows

P.S. Please don’t tell me not to run pip install commands this way. I have a specific use case and it will be a waste of time to argue about this. This is a general issue that extends far beyond pip, and I’m sure being able to parse progress bars from scripts is helpful to more people than just me.

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5

github_iconTop GitHub Comments

1reaction
willmcgugancommented, Jun 7, 2022

Potentially. See https://github.com/pypa/pip/issues/10909 for background

1reaction
willmcgugancommented, Jun 6, 2022

Rich will strip anything dynamic when it detects it is not writing to a console.

If you don’t want this you can force it to behave as a terminal with force_terminal=True. However, since you don’t control that, you should consider using a psuedo-terminal rather than a subprocess.

Read more comments on GitHub >

github_iconTop Results From Across the Web

python - Subprocess pipe broken only when running a script
I'm trying to test a small server application using python. The application is working a bit like a remote shell - it takes...
Read more >
The subprocess Module: Wrapping Programs With Python
In this tutorial, you'll learn how to leverage other apps and programs that aren't Python, wrapping them or launching them from your Python ......
Read more >
How to pass data outside process for zenity progress?
This allows my main script to work in a linear manner, without worrying about passing data from a subprocess, because there is no...
Read more >
Advanced Bash-Scripting Guide
A Simple Implementation of a Progress Bar; 32-8. ... This applies to regular files, not pipes, symlinks, and certain special files.
Read more >
Python Run External Command And Get Output On Screen or ...
Explains how to call an external program using a python script and retrieve the program output & return code/exit status.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found