Codes printed to terminal when using typeahead with FinalTerm sequences

Consider the finalterm-shell-integration.py example. Make the following change to it

--- a/examples/finalterm-shell-integration.py
+++ b/examples/finalterm-shell-integration.py
@@ -8,7 +8,7 @@ from __future__ import unicode_literals
 from prompt_toolkit import prompt
 from prompt_toolkit.token import Token
 import sys
-
+import time

 BEFORE_PROMPT = '\033]133;A\a'
 AFTER_PROMPT = '\033]133;B\a'
@@ -29,8 +29,9 @@ def get_prompt_tokens(cli):


 if __name__ == '__main__':
-    answer = prompt(get_prompt_tokens=get_prompt_tokens)
-
-    sys.stdout.write(BEFORE_OUTPUT)
-    print('You said: %s' % answer)
-    sys.stdout.write(AFTER_OUTPUT.format(command_status=0))
+    while True:
+        answer = prompt(get_prompt_tokens=get_prompt_tokens)
+        time.sleep(2)
+        sys.stdout.write(BEFORE_OUTPUT)
+        print('You said: %s' % answer)
+        sys.stdout.write(AFTER_OUTPUT.format(command_status=0))

Then run the script and type

hello
hello
hello
hello

several times without waiting for the prompt. Eventually, you’ll see something like this:

Say something: # hello
^[[45;1Rello
You said: hello

The ^[[45;1R is printed before the output. I don’t understand what that escape code is exactly. It doesn’t look like one of the FinalTerm codes. I believe the final term codes themselves are being printed correctly, as the “select output” feature in iTerm2 works correctly (it selects everything, including the ^[[45;1R).

[asmeurer]

As far as I can tell this is fixed in 2.0. I think somewhere between the new typeahead code and the better CPR handling it got fixed (I can bisect if you are interested).

I think I figured out the issue with PipeInput specifically (see https://github.com/jonathanslenders/python-prompt-toolkit/issues/610#issuecomment-385522974). Basically the stdout Output class assumes the input can respond to a CPR request, and has no way to know if it is PipeInput. Perhaps PipeInput should get a faked CPR request responder to deal with this? The real problem seems to be the disconnect between Input and Output classes in the code, even though CPR requests and responses inherently link the two (and require they both be a real TTY). At any rate, for now, I’ve come the conclusion that PipeInput should never be used for interactive purposes (only for things like unit tests). I’m still working on translating my code to 2.0 so I can’t say yet if my issues fully exist there.

One thing I couldn’t figure out though is what exactly the CPR responses even do. I commented out the CPR request code, and everything seemed to still act normal in my application. I also noticed that 2.0 has some code to automatically stop sending CPR codes if it notices they are not being responded to. So I’m curious exactly what I should expect to stop working if CPR codes are disabled. And if it’s nothing I care about if I could just disable them myself.

[max-sixty]

For more info, my ipython_config.py consists of:

import sys
from prompt_toolkit.key_binding.vi_state import InputMode, ViState


def get_input_mode(self):
    return self._input_mode


def set_input_mode(self, mode):
    shape = {InputMode.NAVIGATION: 2, InputMode.REPLACE: 4}.get(mode, 6)
    raw = u'\x1b[{} q'.format(shape)
    if hasattr(sys.stdout, '_cli'):
        out = sys.stdout._cli.output.write_raw
    else:
        out = sys.stdout.write
    out(raw)
    sys.stdout.flush()
    self._input_mode = mode


ViState._input_mode = InputMode.INSERT
ViState.input_mode = property(get_input_mode, set_input_mode)
c.TerminalInteractiveShell.editing_mode = 'vi'

Partly from https://github.com/jonathanslenders/python-prompt-toolkit/issues/192#issuecomment-380654372