Normalizing gpi_embed and GPI_EXTRA libraries

This started with https://github.com/cocotb/cocotb/pull/1293#discussion_r381814190, where I considered getting rid of embed_sim_event from the embed interface and adding a register_sim_event_callback to the GPI.

This got me think about what else we could move from the embed interface to the GPI via register_* functions. And the answer is everything (see edit). There really only needs to be an init or entry point function on the embed interface, from there it can register event and cleanup callbacks with the GPI. Sidebar question: Is there some reason that embed_init_python is a separate function and not included in embed_sim_init?

With only an entry point function, the embed interface can go away entirely, because it is essentially equivalent to what is done for GPI_EXTRA. gpi_embed can simply be one such library. The only change to that interface would require we pass argc and argv to those extra library calls, which should already be done anyways.

With all of these changes in place, GPI is a standalone product, and GPI_EXTRA is a mechanism for custom extensions. We could conceivably reuse GPI for other language extensions. There was someone on the gitter a few weeks ago asking for a Java port. This would now be conceivable.

This issue is in place mostly to gather thoughts on the soundness of the idea before I go implement it.

EDIT

There are 4 functions on the embed interface (they can be found in embed.h):

• embed_init_python: this would be merged into the entry function, currently called embed_sim_init.
• embed_sim_cleanup: this would no longer be on the interface, but be registered with the GPI using a new register_sim_cleanup_callback function.
• embed_sim_init: this would be the entry function, the name may change, but this will remain.
• embed_sim_event: this would no longer be on the interface, but be registered with the GPI using a new register_sim_event_callback function.

[ktbarrett]

My plan involves a pretty large refactor of the GPI:

• Move product and version info from embed_sim_init call to getters in simulator module. The idea is to make embed_sim_init look more like main. This also lift responsibility out of C into Python (#1737).
  • gpi_get_sim_product
  • gpi_get_sim_version
• Merge parts of gpi_log into gpi and others into simulatormodule. Provide an interface on the GPI to register custom logging handlers. This will allow us to lift that code into Python.
  • gpi_log: uses active logger
  • gpi_log_native: uses native logger
  • gpi_set_log_handler: register a log handler
  • gpi_reset_log_handler: reset active log handler to native logger
  • gpi_native_logger_set_level: set the log level of the native logger
  • simulatormodule will define the C binding code allow users to register Python functions for that logger, which is how we will inject the Python logger into the GPI
• Remove embed interface and replace it with a registration system. Load GPI users with GPI_EXTRA. This will allow multiple runtime-specified GPI users instead of a single hardcoded one. Allowing us to better support extensions (like #1217) and other non-Python GPI bindings.
  • gpi_register_startup_callback: like embed_sim_init, main like function to call at startup (after all users are loaded and elab has finished)
  • gpi_register_event_callback: like embed_sim_event
  • gpi_register_shutdown_callback: like embed_sim_end, called when the simulation finishes.
  • GPI_EXTRA used to register GPI users
  • What is gpi_embed will become a GPI user
  • wrappers can be written in simulatormodule, allowing Python code to run arbitrary commands on shutdown or simulator event (like cocotb._sim_event)
• Move a lot of the code in gpi_embed up into Python and add custom entry point. At this point gpi_embed’s only job is to start the Python interpreter and run the entry function.

Most of this work is already done sitting in a branch that needs to be rebased, cleaned up, tests added, and one bug (related to shutdown behavior) fixed. I’ll bring it in (in small pieces) after I finish Python 2 cleanup/concurrency work.

[marlonjames]

We’ll also want to change gpi_logging behavior to register a log handler (*_v style). Then gpi_log will pass log calls to handler and check the return value, falling back to gpi_log_native_v if the handler returns an error. I was working on this before @eric-wieser’s gpi_log changes, which should make this much easier now.