`patch_stdout` lines printed out of order

Lines printed from an inputhook might be printed out of order when doing patch_stdout.

Example to illustrate:

from __future__ import unicode_literals
from prompt_toolkit.shortcuts import prompt, create_eventloop
from time import sleep

x = 0
def inputhook(context):
    global x

    print('%d before sleep' % x)
    x = x + 1
    sleep(1)
    print('%d after sleep' % x)
    x = x + 1

prompt('> ',
       eventloop=create_eventloop(inputhook=inputhook),
       patch_stdout=True)

Example output (the order will be different between runs):

0 before sleep
1 after sleep
2 before sleep
3 after sleep
5 after sleep
4 before sleep

Expected numbers to be 0, 1, 2, 3, 4, 5, but instead they are 0, 1, 2, 3, 5, 4.

Running MacOS 10.11.6 and iterm2 3.0.15.

Observations so far:

• the behavior is the same on python 2.7 and python 3.5 • the behavior is (unsurprisingly) the same even if use sys.stdout.write and sys.stdout.flush() instead of print, however, if I write without a newline, lines are in order.

[atherra]

Seeing this as well, on MacOS and Linux, appears to be caused by randomization of high priority tasks in eventloop.PosixEventLoop.run:

random.shuffle(tasks)
random.shuffle(low_priority_tasks)

For the time being, I’m working around this by monkey-patching _StdoutProxy’s _do method to call run_in_terminal directly instead of passing it to eventloop.call_from_executor - but I’m not sure whether this may cause any undesirable side effects.

[jonathanslenders]

Wow, interesting… This should be a bug. Thanks for reporting and the code example! That’s very helpful.