[BUG] RichHandler: jumbled output with existing module logger and logging config from file

Read the issues I read through the existing issues and found some that might be related, but not quite and a bit outdated: #69, #87 and bonus points module name (in the example below "this.module") #850.

update: Just noticed that the module name can be included if e.g. the simple formatter is used for the handler rich (see log_rich.yml below).

Describe the bug The logging and progress bar updates get jumbled in the console when using an existing module logger which is configured externally to use the rich.logging.RichHandler, see the output:

To Reproduce Adapted from the packaged python -m rich.progress plus external logging config file log_rich.yml:

version: 1
formatters:
  simple:
    format: '%(asctime)s %(name)s %(levelname)s %(message)s'
    datefmt: '%Y-%m-%dT%H:%M:%S'
handlers:
  console:
    class: logging.StreamHandler
    formatter: simple
    stream: ext://sys.stdout
  rich:
    class: rich.logging.RichHandler
  file:
    class: logging.FileHandler
    formatter: simple
    filename: this.log
    mode: a
loggers:
  this.module:
    level: DEBUG
    handlers: [ rich, file ]

The reproducer manual_logging_rich.py:

import logging
from logging.config import dictConfig
import random
import time

from rich.console import Console
from rich.panel import Panel
from rich.progress import (
    BarColumn,
    Progress,
    SpinnerColumn,
    TextColumn,
    TimeElapsedColumn,
    TimeRemainingColumn,
)
from rich.rule import Rule
from rich.syntax import Syntax
from rich.table import Table
import yaml

LOGGER = logging.getLogger("this.module")


if __name__ == "__main__":
    _log = "log_rich.yml"
    try:
        with open(_log, "rt") as f:
            config = yaml.load(f.read(), yaml.FullLoader)
            dictConfig(config)
    except FileNotFoundError:
        LOGGER = logging.getLogger()
        LOGGER.setLevel(logging.DEBUG)

    syntax = Syntax(
        '''def loop_last(values: Iterable[T]) -> Iterable[Tuple[bool, T]]:
    """Iterate and generate a tuple with a flag for last value."""
    iter_values = iter(values)
    try:
        previous_value = next(iter_values)
    except StopIteration:
        return
    for value in iter_values:
        yield False, previous_value
        previous_value = value
    yield True, previous_value''',
        "python",
        line_numbers=True,
    )

    table = Table("foo", "bar", "baz")
    table.add_row("1", "2", "3")

    progress_renderables = [
        "Text may be printed while the progress bars are rendering.",
        Panel("In fact, [i]any[/i] renderable will work"),
        "Such as [magenta]tables[/]...",
        table,
        "Pretty printed structures...",
        {"type": "example", "text": "Pretty printed"},
        "Syntax...",
        syntax,
        Rule("Give it a try!"),
    ]

    from itertools import cycle

    examples = cycle(progress_renderables)

    console = Console(record=True)

    with Progress(
        SpinnerColumn(),
        TextColumn("[progress.description]{task.description}"),
        BarColumn(),
        TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
        TimeRemainingColumn(),
        TimeElapsedColumn(),
        console=console,
        transient=True,
    ) as progress:
        LOGGER.debug("existing log message")

        task1 = progress.add_task("[red]Downloading", total=1000)
        task2 = progress.add_task("[green]Processing", total=1000)
        task3 = progress.add_task("[yellow]Thinking", total=1000, start=False)

        LOGGER.debug("existing log message")

        while not progress.finished:
            progress.update(task1, advance=0.5)
            progress.update(task2, advance=0.3)
            LOGGER.debug("existing log message")
            time.sleep(0.01)
            if random.randint(0, 100) < 1:
                progress.log(next(examples))

Platform Running on Windows 10.18363, PowerShell 6.2.2 within Windows Terminal 1.9.1942.0.

Diagnose

❯ python -m rich.diagnose
╭───────────────────────── <class 'rich.console.Console'> ─────────────────────────╮
│ A high level console interface.                                                  │
│                                                                                  │
│ ╭──────────────────────────────────────────────────────────────────────────────╮ │
│ │ <console width=209 ColorSystem.TRUECOLOR>                                    │ │
│ ╰──────────────────────────────────────────────────────────────────────────────╯ │
│                                                                                  │
│     color_system = 'truecolor'                                                   │
│         encoding = 'utf-8'                                                       │
│             file = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'> │
│           height = 28                                                            │
│    is_alt_screen = False                                                         │
│ is_dumb_terminal = False                                                         │
│   is_interactive = True                                                          │
│       is_jupyter = False                                                         │
│      is_terminal = True                                                          │
│   legacy_windows = False                                                         │
│         no_color = False                                                         │
│          options = ConsoleOptions(                                               │
│                        size=ConsoleDimensions(width=209, height=28),             │
│                        legacy_windows=False,                                     │
│                        min_width=1,                                              │
│                        max_width=209,                                            │
│                        is_terminal=True,                                         │
│                        encoding='utf-8',                                         │
│                        max_height=28,                                            │
│                        justify=None,                                             │
│                        overflow=None,                                            │
│                        no_wrap=False,                                            │
│                        highlight=None,                                           │
│                        markup=None,                                              │
│                        height=None                                               │
│                    )                                                             │
│            quiet = False                                                         │
│           record = False                                                         │
│         safe_box = True                                                          │
│             size = ConsoleDimensions(width=209, height=28)                       │
│        soft_wrap = False                                                         │
│           stderr = False                                                         │
│            style = None                                                          │
│         tab_size = 8                                                             │
│            width = 209                                                           │
╰──────────────────────────────────────────────────────────────────────────────────╯

❯ python -m rich._windows
platform="Windows"
WindowsConsoleFeatures(vt=True, truecolor=True)

❯ conda env export | grep rich
  - rich=10.6.0=py38haa244fe_0

Did I help?

If I was able to resolve your problem, consider sponsoring my work on Rich, or buy me a coffee to say thanks.

[fleimgruber]

Thanks for your thoughts, much appreciated.

I’m guessing where the config says “class” you can supply a callable that returns a RichHandler constructed with your console object.

In this case I am not sure where to instantiate the console initially. I guess in the mentioned callable? But then how to pass that to Progress? It would have to be picked out of the RichHandler in turn, I guess. A better approach might be to define a console object on module level (where the callable is defined) and that same console object could be imported and used for Progress. I think the latter option would work and be cleanest?

Failing that you could extend RichHandler and cusomize init to set the console there.

Ah I guess this extended handler class would also import a shared module scope console and use that in the init? And the extended class would then be used in the dictconfig?

I will experiment some more and try one of the suggested approaches, thanks!

And also thanks for your awesome work and commitment on all your public libraries!

[willmcgugan]

Yeah, record=True is only necessary if you want to export text / html.

As long as the handler and progress use the same console instance it should keep the output looking nice.