question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Arista echoes command even though it doesn't process it (early in login process)

See original GitHub issue

We are seeing what seems to be a race condition with ssh logins on arista devices.

Here’s the symptom:

(Channel opened)
DEBUG:netmiko:write_channel: b'\n'
DEBUG:netmiko:write_channel: b'terminal width 511\n'
DEBUG:netmiko:read_channel: Last login: Mon Aug 15 20:10:51 2022 from 10.33.118.14


--- Banner text elided ---

DEBUG:netmiko:read_channel: terminal width 511

DEBUG:netmiko:read_channel:
DEBUG:netmiko:read_channel:
DEBUG:netmiko:read_channel:
DEBUG:netmiko:read_channel:
.
.
.
DEBUG:netmiko:read_channel:
DEBUG:netmiko:read_channel:
DEBUG:netmiko:read_channel:
terminal width 511

DEBUG:netmiko:read_channel:
DEBUG:netmiko:read_channel:
DEBUG:netmiko:read_channel: hostname#
DEBUG:netmiko:read_channel:
.
.
.
20 seconds of read_channel:
.
.
.
DEBUG:netmiko:read_channel:write_channel: b'\n'
DEBUG:netmiko:read_channel:
DEBUG:netmiko:read_channel:
hostname#
DEBUG:netmiko:Pattern found: ([>\#])
hostname#
DEBUG:netmiko:write_channel: b'exit\n'

Pattern not detected: 'Width set to' in output.

Things you might try to fix this:
1. Adjust the regex pattern to better identify the terminating string. Note, in
many situations the pattern is automatically based on the network device's prompt.
2. Increase the read_timeout to a larger value.

You can also look at the Netmiko session_log or debug log for more information.

Notice that the term len 511 string was sent before the prompt was displayed.

Here’s an example showing that this can lead to data loss:

$ ssh hostname
Password: 
Last login: Mon Aug 15 20:19:46 2022 from xxx
 
--- banner message elided ---


hostname# cdefghinjklmopqrstuvwzyz
% Invalid input
hostname# cdefghinjklmopqrstuvwzyz

The input is abcdefghijklmnopqrstuvwxyz\r entered via a keyboard macro immediately after typing return for the password. In this particular case, the first three characters are dropped (I pressed up arrow to see what was actually read by the CLI). If I’m fast, it’s pretty easy for the entire string to be missed. (Sometimes it’s echoed back, but it’s not actually read by the CLI, as can be seen above.)

Looking at the Arista implementation of session_preparation(), it looks like there is no synchronization with the CLI before sending the string to set the terminal width. Indeed, adding a call to

self._test_channel_read(pattern=r"[>#]")

prior to the call to set_terminal_width() resolves the issue. Looking at history, it looks like this was removed as part fo the optimizations in #1960 .

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
tlabergecommented, Aug 15, 2022

We love our vendors. I will open a case with them, too. Thanks for the quick response.

0reactions
ktbyerscommented, Sep 13, 2022

I think this should help with the issue:

https://github.com/ktbyers/netmiko/pull/2944

There still could be problems as Arista is really doing something it shouldn’t do here (i.e. echoing the command when the CLI is not actually ready to process it and also double echo of the command…though it is possible there are other low-level CLI things going on here that I don’t have enough knowledge on).

Read more comments on GitHub >

github_iconTop Results From Across the Web

Arista EOS Hardening Guide
User Role Management. Role based authorization is a method of restricting access to CLI command through the assignment of profiles, which are ...
Read more >
EOS 4.29.0F - Command-Line Interface (CLI) - Arista
The command-line interface (CLI) is one tool for controlling the switch and displaying information about its status and configuration. This chapter describes ...
Read more >
IP Locking - UM - EOS User Manual - Arista
Role-based authorization is a method of restricting access to CLI commands through the assignment of profiles, called roles, to user accounts.
Read more >
EVPN Type-5 Routes: IP Prefix Advertisement - Arista Networks
The show ip route <VRF> command can be used to determine which prefixes are ... If there is still a tie at the...
Read more >
EOS 4.29.0F - Managing Display Attributes - Arista
These commands create the login and motd banner shown earlier in this section . ... The switch responds with instructions on entering the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found