[BUG] rich.traceback: no lexer for filename X found

Describe the bug

The rich.traceback doesn’t know how to render tracebacks originating from inside jinja2 templates. The issue is twofold: (1) rich doesn’t recognize the .yaml.j2 extension as jinja2+yaml, and (2) the fallback isn’t to print the file contents without highlighting but instead to simply not print it at all. As such this bug likely also affects other languages to compile to python bytecode, such as dg, Hylang and hissp/hebigo.

Repro:

test.py:

import jinja2, os
from rich.traceback import install
install()
def raiser():
    raise Exception
env = jinja2.Environment(
    loader = jinja2.FileSystemLoader([os.getcwd(), "/"], followlinks=True),
)
env.globals["raiser"] = raiser    
template = env.get_template("test.yaml.j2")
print(template.render())

test.yaml.j2:

foobar:
    something: {{ raiser() }}
    else: {{ 5 + 5 }}

Vanilla traceback:

Traceback (most recent call last):
  File "/home/pbsds/tmp/repro/test.py", line 13, in <module>
    print(template.render())
  File "/usr/lib/python3.10/site-packages/jinja2/environment.py", line 1291, in render
    self.environment.handle_exception()
  File "/usr/lib/python3.10/site-packages/jinja2/environment.py", line 925, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/home/pbsds/tmp/repro/test.yaml.j2", line 2, in top-level template code
    something: {{ raiser() }}
  File "/home/pbsds/tmp/repro/test.py", line 4, in raiser
    raise Exception
Exception

rich.traceback:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│                                                                                                  │
│ /home/pbsds/tmp/repro/test.py:15 in <module>                                                     │
│                                                                                                  │
│   12                                                                                             │
│   13 template = env.get_template("test.yaml.j2")                                                 │
│   14                                                                                             │
│ ❱ 15 print(template.render())                                                                    │
│   16                                                                                             │
│ /usr/lib/python3.10/site-packages/jinja2/environment.py:1291 in render                           │
│                                                                                                  │
│   1288 │   │   try:                                                                              │
│   1289 │   │   │   return concat(self.root_render_func(ctx))  # type: ignore                     │
│   1290 │   │   except Exception:                                                                 │
│ ❱ 1291 │   │   │   self.environment.handle_exception()                                           │
│   1292 │                                                                                         │
│   1293 │   async def render_async(self, *args: t.Any, **kwargs: t.Any) -> str:                   │
│   1294 │   │   """This works similar to :meth:`render` but returns a coroutine                   │
│                                                                                                  │
│ /usr/lib/python3.10/site-packages/jinja2/environment.py:925 in handle_exception                  │
│                                                                                                  │
│    922 │   │   """                                                                               │
│    923 │   │   from .debug import rewrite_traceback_stack                                        │
│    924 │   │                                                                                     │
│ ❱  925 │   │   raise rewrite_traceback_stack(source=source)                                      │
│    926 │                                                                                         │
│    927 │   def join_path(self, template: str, parent: str) -> str:                               │
│    928 │   │   """Join a template with the parent.  By default all the lookups are               │
│                                                                                                  │
│ /home/pbsds/tmp/repro/test.yaml.j2:2 in top-level template code                                  │
│                                                                                                  │
│ no lexer for filename 'test.yaml.j2' found                                                       │
│                                                                                                  │
│ /home/pbsds/tmp/repro/test.py:6 in raiser                                                        │
│                                                                                                  │
│    3 install()                                                                                   │
│    4                                                                                             │
│    5 def raiser():                                                                               │
│ ❱  6 │   raise Exception                                                                         │
│    7                                                                                             │
│    8 env = jinja2.Environment(                                                                   │
│    9 │   loader = jinja2.FileSystemLoader([os.getcwd(), "/"], followlinks=True),                 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯

Platform

Click to expand

What platform (Win/Linux/Mac) are you running on? What terminal software are you using?

Manjaro GNOME, python 3.10.2, lxterminal

I may ask you to copy and paste the output of the following commands. It may save some time if you do it now.

If you’re using Rich in a terminal:

╭─────────────────────── <class 'rich.console.Console'> ───────────────────────╮
│ A high level console interface.                                              │
│                                                                              │
│ ╭──────────────────────────────────────────────────────────────────────────╮ │
│ │ <console width=80 ColorSystem.TRUECOLOR>                                 │ │
│ ╰──────────────────────────────────────────────────────────────────────────╯ │
│                                                                              │
│     color_system = 'truecolor'                                               │
│         encoding = 'utf-8'                                                   │
│             file = <_io.TextIOWrapper name='<stdout>' mode='w'               │
│                    encoding='utf-8'>                                         │
│           height = 24                                                        │
│    is_alt_screen = False                                                     │
│ is_dumb_terminal = False                                                     │
│   is_interactive = True                                                      │
│       is_jupyter = False                                                     │
│      is_terminal = True                                                      │
│   legacy_windows = False                                                     │
│         no_color = False                                                     │
│          options = ConsoleOptions(                                           │
│                        size=ConsoleDimensions(width=80, height=24),          │
│                        legacy_windows=False,                                 │
│                        min_width=1,                                          │
│                        max_width=80,                                         │
│                        is_terminal=True,                                     │
│                        encoding='utf-8',                                     │
│                        max_height=24,                                        │
│                        justify=None,                                         │
│                        overflow=None,                                        │
│                        no_wrap=False,                                        │
│                        highlight=None,                                       │
│                        markup=None,                                          │
│                        height=None                                           │
│                    )                                                         │
│            quiet = False                                                     │
│           record = False                                                     │
│         safe_box = True                                                      │
│             size = ConsoleDimensions(width=80, height=24)                    │
│        soft_wrap = False                                                     │
│           stderr = False                                                     │
│            style = None                                                      │
│         tab_size = 8                                                         │
│            width = 80                                                        │
╰──────────────────────────────────────────────────────────────────────────────╯
rich==11.0.0

[wasi-master]

I’ve found the cause of this problem, the guess_lexer_for_filename method from the pygments.lexers module raises a ClassNotFound upon not being able to find a suitable lexer. And that function is used in the rich Traceback._guess_lexer function:

https://github.com/Textualize/rich/blob/master/rich/traceback.py#L513-L527

Upon getting a error, the traceback printer just prints the error gotten as a string which most of the times prints the error message:

https://github.com/Textualize/rich/blob/master/rich/traceback.py#L625-L627

The error gotten in this case is pygments’ ClassNotFound error with the message no lexer for filename '...' found.

This can be fixed by adding another except statement for handling ClassNotFound and using the text lexer as a fallback

try:
    ...
except ClassNotFound as error:
    yield Syntax(
        code,
        "text",
        theme=theme,
        line_numbers=True,
        line_range=(
            frame.lineno - self.extra_lines,
            frame.lineno + self.extra_lines,
        ),
        highlight_lines={frame.lineno},
        word_wrap=self.word_wrap,
        code_width=88,
        indent_guides=self.indent_guides,
        dedent=False,
    )
except Exception as error:
    ...
else:
    ...

This would create some duplicate code, I’m sure the brilliant minds at @Textualize can figure something out

Applying these changes fixes the issue

cc: @willmcgugan @darrenburns

[darrenburns]

Rich delegates a lot of this stuff to Pygments, which itself doesn’t recognise .yaml.j2 as an extension. If your files have the .sls extension, Pygments will use the YAML+Jinja lexer. Here’s the code from Pygments that does this: https://github.com/pygments/pygments/blob/64e8e05307689fe726ce9df2d5907e9c2fb67405/pygments/lexers/templates.py#L1861

It may be worth opening an issue/PR with Pygments, requesting to add this extension if it’s one that is commonly used - we’d rather not start adding workarounds/logic for this sort of thing on our side.

On the Rich side, we can certainly fix the bug where no text was being output.