Consideration for ANSI output over IPC / "next-gen" remote debugging

Hey @jonathanslenders 😸

I was wondering what you thought about the idea of formalizing an API / configuration that would make it possible run ptk in a remote process and ship output over an IPC channel such that native remote debugging with things like coolio tab completion and syntax highlighting can be a thing; bringing the wonder of Python’s interactivity to distributed systems dev.

---

I’ve got a couple issue written up that cover my research up to this point:

• goodboy/tractor#130
• goodboy/tractor#113

As far as I gather ptk is pretty much the project (even outside python) capable of this feat and so I thought I’d start here 😃

I’ll requote:

The problem

Standard fancy (read human enhanced) debugger repls (including the stdlib’s pdb which uses rlcompleter, and pdb++) rely on libraries such as GNU readline to get things like completion and CLI “editting controls”. There seems to be no way to get these features with readline based systems in a remote debugging context since Python’s use of readline requires that the process is launched under a tty/pty system. Ideally these features are available in such use cases to make debugging of remote systems sane and efficient for the user.

---

I’ll re-summarize a few things we’ve discovered poking around these repos and which we’ve already listed in the above issues:

• options for doing this in a tmux-like app: prompt-toolkit/python-prompt-toolkit#1087
• mention about using IPC to ship ANSI output
• further discussion around capturing the output
• a PR showing how to reroute output to stderr if it’s a tty
• there seems to be test code suggesting ptk can work without being connected to a tty/pty.
• there’s a utility for avoiding the clobbering of stdout that looks to be exactly what I had envisioned for a minimal control repl
• a very old pdb implementation built on ptpython, ptpdb which is likely where we’ll need to start
• ptk has utils for not clobbering stdout which would be super handy for having multiple actors logging while your in the middle of debugging a crash.

Would be greatly interested in what other things I’ve missed and any further input you might have. Also I can add the explicit issue/repo links to that list if this issue is something you deem worth tracking.

---

Also a couple things wrt projects built on top of ptk (or wanting to support it):

• in theory this might work with ipdb (since ipython uses ptk underneath) if ptk was configured properly (it currently doesn’t work any better then the readline options based on my testing in https://github.com/goodboy/tractor/tree/stin_char_relay).
• work with debugger’s that want to move to ptk to get this functionality supported in their initial integration such as with pdbpp in pdbpp/pdbpp#362
• pdb++ discussion on better support for remote debugging

Also fwiw I have tried out ptpdb and I think it’s super slick but, it might just be a bit too much as a default, with the full terminal UI and all (though having that kind of full UI glory is definitely nice to have if the user so chooses).

One of the critical UX things I’m hoping for is the ability to work with a prompt while other asynchronous things are writing to the local tty and not having that prompt get clobbered. I pointed above to a couple things above but I’m hoping you can give me even more pointers on how to accomplish this 🕵️.

Look forward to your thoughts 🏄

[jonathanslenders]

@goodboy : A PromptSession can run in an AppSession. The PromptSession is the object that keeps track of the input history during several input queries, as well as copy/paste registries (and maybe a few other things). The AppSession provides an environment with input/output objects in which all applications can run.

[goodboy]

Dug into ipython (very very) briefly and found where the ptk object entrypoints are:

• instantiating a completer wrapper inside the terminal debugger which is a subtype of prompt_toolkit.completion.Completer
• creation of a PromptSession which is a prompt_toolkit.shortcuts.prompt.PromptSession
  • no idea how this differs from the AppSession mentioned by @jonathanslenders above (yet)