Colored text output renders differently in Screen than in regular terminal

Describe the bug

When using screen.paint with ColouredText and AsciimaticsParser, characters are not always rendered properly. Sometimes the spacing is messed up, other times some of the encoding characters are printed, and other times the text seems to be getting truncated.

To Reproduce

Here is a sample that should show the whitespace and truncating problems. See Additional context section.

from asciimatics.screen import Screen
from asciimatics.parsers import AsciimaticsParser
from asciimatics.strings import ColouredText


def render_code_block(screen, block, x, row):
    cur_row = row
    lines = block.split("\n")

    for line in lines:
        text = ColouredText(
            line,
            AsciimaticsParser(),
        )
        screen.paint(
            text.raw_text,
            x,
            cur_row,
            colour_map=text.colour_map,
        )
        cur_row += 1


def demo(screen):
    line = '\x1b[38;5;28mprint\x1b[39m(\x1b[38;5;124m"\x1b[39m\x1b[38;5;124mThis is a test\x1b[39m\x1b[38;5;124m"\x1b[39m)'

    line2 = "console.log(\x1b[38;5;124m`\x1b[39m\x1b[38;5;124mData is \x1b[39m\x1b[38;5;132;01m${\x1b[39;00mmyFunc(\x1b[38;5;124m'John'\x1b[39m)[\x1b[38;5;241m0\x1b[39m]\x1b[38;5;132;01m}\x1b[39;00m\x1b[38;5;124m`\x1b[39m);"

    while True:
        # rendering this line will show spacing problems
        render_code_block(screen, line, 5, 10)

        # rendering this line will show spacing issues and the truncating issue
        render_code_block(screen, line2, 5, 15)

        ev = screen.get_key()
        if ev in (ord("Q"), ord("q")):
            return
        screen.refresh()


Screen.wrapper(demo)

Expected behavior I expect the text to be painted to the screen like it is printed in the terminal.

Screenshots

Here is a screenshot of what the above sample code gives:

Here’s what I get from a repl when I print the same encoded strings that I’m passing to ColouredText:

If I change the second call to render_code_block to have a 0 instead of a 5 for the x argument, I get to see part of “John”:

System details (please complete the following information):

• OS and version: Linux Mint 20.1
• Python version: 3.8.10
• Python distribution: Default from apt repo for Ubuntu
• Asciimatics version 1.13.0

Additional context

I came across this while trying to add a feature to the present tool. asciimatics is used heavily in that project, and I haven’t come up with a minimum example that shows the encoding characters rendered. I suspect this may have to do with Scenes, Effects, etc. being used. Here’s an example of what that looks like when it happens:

[bkjohnson]

Yes, it’s been addressed. Thank you for the help.

I may take a look at the AsciinemaPlayer if my current PR goes well, thanks for the tip.

[peterbrittain]

I suspect you are using the wrong parser. Looking at your code, you are passing Ansi escape codes into the string, but then using the Asciimatics parser to decode them. This parser just looks for ${c,a,b} escape sequences (as covered in https://asciimatics.readthedocs.io/en/stable/widgets.html#changing-colours-inline).

Try using the AnsiTerminalParser (https://asciimatics.readthedocs.io/en/stable/asciimatics.html#asciimatics.parsers.AnsiTerminalParser) instead.