Problems with expect_string on new Cisco 9100 series APs

Hi, I’ve reviewed #941 and a couple other issues, and this doesn’t appear to be related, so I’m opening a new issue. I have also reviewed the example uses of expect_string, send_command_timing, and troubleshot this quite a bit on my own but I can’t figure out what is going wrong.

I am using Netmiko version 3.3.0.

Problem Summary

I’m trying to change the hostname on a Cisco 9130AX AP, and part of the process involves restarting the CAPWAP connection to the WLC. The AP asks to [confirm] the CAPWAP restart, so I have Python send a \n. The AP seems to completely ignore the \n that Netmiko is sending, and the CAPWAP connection never restarts.

The latest Cisco 9100 series APs run a slightly different software than the previous 3700/3800/4800 gen hardware. I feel like this is the core of the issue, but I don’t want to jump to any conclusions yet.

Test Code

Here’s the relevant pieces of my test script. I have excluded the username, password, and IP on purpose.

from netmiko import Netmiko

ap_shell = Netmiko(host=ip, username=AP_USER, password=AP_PASS, device_type='cisco_ios', secret=AP_PASS)
ap_shell.enable()
new_name = 'TEST'
output = ap_shell.send_command(command_string=f"capwap ap hostname {new_name}", strip_prompt=False, strip_command=False)
time.sleep(1)
output += ap_shell.find_prompt()
output += ap_shell.send_command(command_string=f"capwap ap restart", expect_string=r"confirm", strip_prompt=False, strip_command=False)
output += ap_shell.send_command(command_string="\n", strip_prompt=False, strip_command=False)
print(output)

Here is the output from the above code:

capwap ap hostname TEST
Please note that if AP is already associated to WLC,
the new hostname will only reflect on WLC after AP
 dis-associates and rejoins.

AP1416.9D2A.1A80#AP1416.9D2A.1A80#capwap ap restart
Warning: This CLI resets connection with WLC.
Do you want to continue? [confirm]

TEST#

I know that the the CAPWAP restart is not being triggered because I am watching the live logs of the AP, and also checking the CAPWAP uptime on the WLC. I would see the CAPWAP timer restart on the WLC, and the logs would display messages of a DTLS teardown.

Executing the commands manually by a regular SSH connection show the same output in the terminal… but the CAPWAP connection gets reset.

I’ve also tried to use Netmiko to reload the AP, which asks for a confirmation, just like the capwap restart command. Netmiko is failing to issue the confirmation in that case as well. Here’s what the attempt at reloading using Netmiko looks like in my Python interpreter:

>>> ap_shell = Netmiko(host=ip, username=AP_USER, password=AP_PASS, device_type='cisco_ios', secret=AP_PASS)
>>> ap_shell.enable()
'enable\r\nPassword: \r\n\rTEST#'
>>> 
>>> ap_shell.send_command("reload", expect_string=r'confirm')
'Proceed with reload? [confirm]'
>>> ap_shell.send_command('\n')
''
>>> 

I’ve also tried using write_channel('\n') instead of send_command('\n'). Neither method seems to work.

I read here in this blog post that:

Netmiko determines the current prompt by sending an ‘enter’ right before the command is sent.

… so I’ve also tried with auto_find_prompt=False. No different.

Any ideas how I can troubleshoot this further to figure out what’s going on? Or, do you see an obvious mistake that I overlooked?

[ktbyers]

Does it work if for the last <enter> you do the following:

output = ap_shell.send_command(command_string=f"capwap ap restart", expect_string=r"confirm", strip_prompt=False, strip_command=False)
ap_shell.write_channel("\r\n")
time.sleep(1)
output += ap_shell.read_channel()

Also, you might want to add the session_log so we can see if we can figure out what is going on (of course, maybe you are already doing this):

ap_shell = Netmiko(host=ip, username=AP_USER, password=AP_PASS, device_type='cisco_ios', secret=AP_PASS, session_log="output.txt")

This will create a file named “output.txt” in your current directory.

Regards,

Kirk

[David00]

Kirk,

That worked - thanks so much! Here’s the contents of output:

>>> print(output)
capwap ap restart
Warning: This CLI resets connection with WLC.
TEST#u want to continue? [confirm]
TEST#

Thanks for letting me know about session_log. I’ve been using Netmiko for awhile now (you had previously helped me with issue #1184), and I’ve never noticed the session_log. That will be helpful moving forward.

Do you know why that specific combination of \r\n with write_channel does the trick? I just tested with send_command('\r\n') and it does not work. I understand that write_channel is much lower level, but I don’t know what send_command is doing differently.