cmd_verify fails when the echoed command contains backspaces

Sometimes network devices will insert backspaces into the echoed command and this causes cmd_verify to fail. Eg: ‘file list /var/tmp detail’ on a Junos device will sometime echo back as ‘file list /var/tmp ^H^H^Hdetail’

If control codes could be automatically handled, it would greatly improve the pattern matching reliability.

Here’s a function I wrote to remove control characters from the output of a variety of different network device OSes:

def handle_ctrl_chars(line: str) -> str:
    """
    Process ANSI/ASCII control characters in a line of data
    and return it with them all removed.
    This is not exhaustive - it just handles control sequences I've observed output from network devices.
    """

    # Handle ANSI Erase Line (esc [ K)
    line = re.sub(r"^.*\x1b\[K", "", line, re.MULTILINE)

    # Handle backspaces (ascii 0x08)
    while re.search(r"[^\x08]\x08", line):
        line = re.sub(r"[^\x08]\x08", "", line)
    line = re.sub(r"\x08", "", line)

    # Remove ANSI color sequences
    line = re.sub(r"\x1b\[[\d;]*m", "", line)

    # Remove all control characters below SPACE except for (horizontal) TAB
    # which we convert to four spaces
    line = re.sub(r"\011", "    ", line)
    line = re.sub(r"[\000-\037]", "", line)

    return line

@pytest.mark.parametrize(
    "data, expected, dscr", [
        ("", "", "Empty string"),
        ("abc", "abc", "Normal characters"),
        ("abc\x1b[K", "", "ANSI Erase Line"),
        ("abc\x08", "ab", "Backspace"),
        ("abc\x08\x08", "a", "Backspaces"),
        ("\x08\x08", "", "Backspaces at the front of a line"),
        ("A\x1b[93;41mB", "AB", "Remove ANSI color"),
        ("A\x1b[93;41mB\x1b[0m", "AB", "Remove several ANSI color"),
        ("A\011B", "A    B", "Replace tab with four spaces"),
        ("A\001B\003C", "ABC", "Remove control characters"),
    ])
def test__handle_ctrl_chars(data, expected, dscr):
    assert handle_ctrl_chars(data) == expected

[campbsb]

I’ve put some diagnostics into my codebase, and we seem to have ceased using those devices which responded using ANSI characters. However, aside from the buggy JUNOS version, we also see regular uses of backspaces in the echoed commands by BIG-IP F5 TMOS devices. Trialling with netmiko shows that its setting of the line width also handles these boxes.

So - all good, thanks for some really useful code.

[campbsb]

Thanks for responding. That ANSI escape code stripping is pretty good - but could be improved by having the erase line codes actually erase the line! Otherwise you get lots of duplication in the output where the network device re-echoes what it think’s it’s erased. I forget which OS this was for - I apply it to all of them now! I also note that the strip_backspaces function just removes them, instead of applying them.

For my backspace issue, I’ve attached an anonymised session log here - sessionlog.txt

The Junos version is 20.3R1-S3.12-EVO We can see that the ‘set cli screen-width 511’ is applied and acknowledged, so it seems to be a bug in this Junos version.

It occurs reproducibly on multiple Juniper boxes running this Junos version, but does not appear on earlier or later Junos versions.

So - the underlying cause is an issue with Junos 20.3R1-S3.12-EVO not respecting the ‘set cli screen-width 511’, but I’d still like netmiko to handle backspaces - simply stripping them out is insufficient. And it has to handle them before trying to match the command prompt.