question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. Itย collects links to all the places you might be looking at while hunting down a tough bug.

And, if youโ€™re still stuck at the end, weโ€™re happy to hop on a call to see how we can help out.

[BUG] Progress(): Terminal size mis-detected if run in multiprocessing sub-process

See original GitHub issue

Describe the bug Using multiprocessing to run a sub process, initializing Progress() under that process fails to detect the terminal size, using the fallback (80, 25) instead.

To Reproduce Sample code:

from multiprocessing import Process, Queue

from rich.console import Console
from rich.progress import Progress


def sub_process():
    pre_console = Console()
    print(f"Sub process PRE console size: {pre_console.size}\n")

    with Progress() as progress:
        print(f"Sub process progress.console.size: {progress.console.size}")

        console = Console()
        print(f"Sub process console size: {console.size}\n")


if __name__ == "__main__":
    with Progress() as main_progress:

        print(f"Main progress.console.size: {main_progress.console.size}")

        main_console = Console()
        print(f"Main console size: {main_console.size}\n")

        q = Queue()
        process = Process(target=sub_process)
        process.start()
        process.join()

Output is:

Main progress.console.size: ConsoleDimensions(width=151, height=45)
Main console size: ConsoleDimensions(width=151, height=45)

Sub process PRE console size: ConsoleDimensions(width=151, height=45)

Sub process progress.console.size: ConsoleDimensions(width=80, height=25)
Sub process console size: ConsoleDimensions(width=80, height=25)

-> Note the difference between the 3 variants:

  • instances in main module (correct terminal size)
  • instance in sub process, before Progress() is called (correct)
  • instances in sub process, after Progress() is called (fallback)

Platform Macos 11.4 (Big Sur) iTerm 2 python 3.9.4

Diagnose I may ask you to cut and paste the output of the following commands. It may save some time if you do it now.

rich.diagnose

โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ <class 'rich.console.Console'> โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ A high level console interface.                                                  โ”‚
โ”‚                                                                                  โ”‚
โ”‚ โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ โ”‚
โ”‚ โ”‚ <console width=151 ColorSystem.TRUECOLOR>                                    โ”‚ โ”‚
โ”‚ โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ โ”‚
โ”‚                                                                                  โ”‚
โ”‚     color_system = 'truecolor'                                                   โ”‚
โ”‚         encoding = 'utf-8'                                                       โ”‚
โ”‚             file = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'> โ”‚
โ”‚           height = 45                                                            โ”‚
โ”‚    is_alt_screen = False                                                         โ”‚
โ”‚ is_dumb_terminal = False                                                         โ”‚
โ”‚   is_interactive = True                                                          โ”‚
โ”‚       is_jupyter = False                                                         โ”‚
โ”‚      is_terminal = True                                                          โ”‚
โ”‚   legacy_windows = False                                                         โ”‚
โ”‚         no_color = False                                                         โ”‚
โ”‚          options = ConsoleOptions(                                               โ”‚
โ”‚                        size=ConsoleDimensions(width=151, height=45),             โ”‚
โ”‚                        legacy_windows=False,                                     โ”‚
โ”‚                        min_width=1,                                              โ”‚
โ”‚                        max_width=151,                                            โ”‚
โ”‚                        is_terminal=True,                                         โ”‚
โ”‚                        encoding='utf-8',                                         โ”‚
โ”‚                        justify=None,                                             โ”‚
โ”‚                        overflow=None,                                            โ”‚
โ”‚                        no_wrap=False,                                            โ”‚
โ”‚                        highlight=None,                                           โ”‚
โ”‚                        markup=None,                                              โ”‚
โ”‚                        height=None                                               โ”‚
โ”‚                    )                                                             โ”‚
โ”‚            quiet = False                                                         โ”‚
โ”‚           record = False                                                         โ”‚
โ”‚         safe_box = True                                                          โ”‚
โ”‚             size = ConsoleDimensions(width=151, height=45)                       โ”‚
โ”‚        soft_wrap = False                                                         โ”‚
โ”‚           stderr = False                                                         โ”‚
โ”‚            style = None                                                          โ”‚
โ”‚         tab_size = 8                                                             โ”‚
โ”‚            width = 151                                                           โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ

rich._windows

platform="Darwin"
WindowsConsoleFeatures(vt=False, truecolor=False)

pip freeze | grep rich

rich==10.2.2

** Workaround ** Initalize a Console() with width and height from shutil.get_terminal_size(), then initialize Progress() with that console:

    terminal_size: os.terminal_size = shutil.get_terminal_size()
    console = Console(width=terminal_size.columns, height=terminal_size.lines)
    progress = Progress(console=console)

It would seem that Console() only uses shutil.get_terminal_size() under Windows. Otherwise it uses os.get_terminal_size() for stdin or stdout - but I guess (I havenโ€™t fully explored your code, sorry) that the redirects that are enabled by Progress() / Live(), coupled with the multiprocessing, somehow mess that up. Perhaps shutil could be tried first, regardless of platform?

Thanks for an awesome library!

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5

github_iconTop GitHub Comments

1reaction
willmcgugancommented, Jul 13, 2021

Please try 10.6.0 It will use sys.stdin etc. so it should still work even when wrapper.

0reactions
mboquiencommented, Jul 13, 2021

I ran into the same issue and the new release appears to have solved it. Thanks a lot!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Rapidly printing to stderr from separate process causes ...
Describe the bug When the stderr stream contains a lot of lines in rapid ... from separate process causes overwriting of progress bar...
Read more >
Show the progress of a Python multiprocessing pool ...
My personal favorite -- gives you a nice little progress bar and completion ETA while things run and commit in parallel. from multiprocessing...
Read more >
Python Multiprocessing Pool: The Complete Guide
The process pool provides a parallel version of the built-in map() function for issuing tasks. For example: 1. 2. 3. 4.
Read more >
Why your multiprocessing Pool is stuck (it's full of sharks!)
You're using multiprocessing to run some code across multiple processes, and it justโ€”sits there. It's stuck.
Read more >
The subprocess Module: Wrapping Programs With Python
It has a main() function that returns an integer. When this program runs, the operating system will interpret its execution as successful sinceย ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found