Dev: hot-reloading newm's own source?

First, thanks! newm is both innovative and (together with pywm) probably the easiest starting point that now exists for hackable wayland WM 👏

Do you have a recipe for reloading newm source itself?

Things I found out today:

• I can reload() modules and log stuff from config file. Poor man’s REPL — edit config, press hot-key, see output in tail -f $HOME/.cache/newm_log | grep -v 'GL_INVALID_VALUE|TIMER|DEBUG|wm_output.c' (I tweak the command to whatever I consider noise at the moment) 🤣

• It’s convenient to do things in on_reconfigure function so the output appears closer to the end. (OTOH if I were hacking on logic such as _setup_widgets I’d maybe want to reload it before it runs? EDIT: see below, this proved better)

• As usual in Python, reload()ing modules is not enough when other modules:

  1. already have instances of them.
  2. worse, have code running in threads!
  3. have copies e.g. via from mod import var, Class.

  At least for widgets, (1) (2) are luckily covered by layout.py stopping threads, destroying and re-creating instances.

  (3) is a problem, specifically from .widget import TopBar, BottomBar, Background, Corner, FocusBorders means layout modules would still use references to previous class object to re-create instances. I now came up with a quick hack in config file that partially monkey-patches such copies, enough to let me modify widgets appearance and seeing the results without newm restart:

def on_reconfigure():
    logger.warning('~' * 120)
    
    import sys, importlib

    mod_names = ['newm.widget.bar', 'newm.widget.focus_border', 'newm.layout']
    for mod_name in mod_names:
        if mod_name in sys.modules:
            importlib.reload(sys.modules[mod_name])

    for mod_name in mod_names:
        for name, obj in vars(sys.modules[mod_name]).items():
            source_mod_name = getattr(obj, '__module__', None)
            if source_mod_name and source_mod_name != mod_name and source_mod_name in sys.modules and 'newm' in source_mod_name:
                current_obj = getattr(sys.modules[source_mod_name], name, None)
                if type(obj) == type(current_obj) and obj is not current_obj:
                    logger.warning('RELOAD PATCHING %s.%s \t<- %s = %r', mod_name, name, source_mod_name, current_obj)
                    setattr(sys.modules[mod_name], name, current_obj)
   
    logger.warning('_' * 120)

I’m sure a better stuff is possible, there are auto-reload modules that do much more intelligent patching…
Do you have a ready setup that you’re using? Or do you always start-newm?

[CRAG666]

This is my configuration if it is useful https://github.com/CRAG666/dotfiles/blob/master/config/newm/configV3.py

[cben]

if you are asking how it works without my kludge:

newm’s current config reload flow

• The shortcut or newm-cmd update-config trigger this:
  https://github.com/jbuchermn/newm/blob/d5831bbca8815fa77c14616c778d3398116b0808/newm/layout.py#L1212-L1217
  • Your config file is reloaded. Top-level code is executed immediately; can also define callbacks for later.
  • it calls _setup()
    • which calls _setup_widgets() which destroys and recreates most Widget instances.
    • it also re-creates some other stuff.
    • This part: https://github.com/jbuchermn/newm/blob/d5831bbca8815fa77c14616c778d3398116b0808/newm/layout.py#L526-L528 calls into pywm setting _pending_config attr that will be sent to C next time it calls _update callback.
  • self.damage() will cause process() to run again and recompute down_state. @jbuchermn would it be useful to self.damage(propagate=True) instead to trigger this pretty much everywhere?
  • After all this, your on_reconfigure() function, if defined, is executed.

Additionally, all config params are used like this:

conf_enable_dbus_gestures = configured_value("gestures.dbus.enabled", True)

which returns a callable; wherever code wants to actually use the value it calls it e.g. if conf_enable_dbus_gestures(): so it always gets a fresh value.
And config params that specify a callback get used like conf_foo()(args).