Support remote Python debugging

After thinking about this some more, this approach locks cocotb into debuggers that use the Debug Adapter Protocol such as VSCode or Theia (used by GitPod). There are a variety of ways to do remote debugging:

• VSCode/GitPod uses debugpy.
• Pycharm Professional uses pydevd-pycharm.
• PyDev in Eclipse uses plain pydevd.
• Our docs recommend remote-pdb.

This may be a good candidate for cocotb extensions that provide a setuptools entry point that can be selected by COCOTB_PY_ATTACH.

So for example, a new extension cocotbext-debugpy could provide an entry point:

setup(
    name = 'cocotbext-debugpy',
    entry_points = {"cocotb.remote_debugger": "debugpy = cocotbext.debugpy:debug_attach"}
    ...
)

And the user would select it with COCOTB_PY_ATTACH=debugpy,localhost:5678

_Originally posted by @garmin-mjames in https://github.com/cocotb/cocotb/pull/2103#issuecomment-705050127_

---

This issue is to discuss the best approach to support remote debugging.

[ktbarrett]

1. Is supporting remote debugging on Python initialization desirable?

Yes IMO.

1. If so, can we settle on an approach to enable it?

Of course!

I like the simplicity of a single environment variable, but it may make sense to split it up.

• COCOTB_PY_DEBUGGER - debugpy, pydevd-pycharm, etc
• COCOTB_PY_ATTACH - host:port or just port if localhost

The number of point solutions and environment variables we are using for those point solutions is already a lot IMO, and adding more is only going to decrease maintainability. I think it’s time to step back and abstract the problem. To me the best solution to abstract all the behaviors we want at startup, as well as support user customization points, as “custom entry points” (#1225). To me, these are more of a startup routine. Attaching a Python debugger can be one of those startup routines, and so could other existing aspects of _initialize_testbench (like enabling coverage).

For example, in your PR you have encapsulated the behavior into a function. Now we can do COCOTB_STARTUP_ROUTINES += ',cocotb:_check_debug_attach'. The remote debugger initialization could live in this repo, or not. Whether it can exist or not is then divorced from everyone else’s approval. I think this is the path forward for new features that must run during initialization. There are also a host of other issues that can be solved using this generalized approach.

P.S. Maybe there is a way to support passing simple literal values safely to these routines? So that you don’t even need the additional environment variable for the port #.

[eric-wieser]

@garmin-mjames: Note in principle that cocotb could support only entry_points, and we could provide a cocotb-plugin-hacks package that provides an entry point that reads COCOTB_STARTUP_ROUTINES (or vice versa)