v8.x.x?

Description

This isn’t really a feature request per-se, its more a discussion about python-semantic-release, the development experience and its continued evolution.

Story time: I added a couple of comments onto an issue a few months ago (#423) regarding support for multiple branches. Since then I can see that #485 was opened with the same request.

Since then I’ve changed my full-time job, etc etc, and I finally found time to start working on a PR yesterday. I love what python-semantic-release and its JS counterpart set out to do, and multibranch support is a must-have for me to integrate it into many projects. I know JS supports this but why should they have all the fun? 😉

However, pretty quickly after I started working on it, I found a couple of things that I realised were really making the experience of working on the feature difficult. So I have changed track and am working on a PR which addresses the following things:

1. There are many levels of indirection within the codebase. I know this generally promotes re-use, but I would like to follow 4-5+ levels of function calls as few times as possible to understand what really happens 🙂
2. The type annotations are informational but not 100% accurate. For example param: str = None. This makes it hard to reason about default values, edge cases to be accounted for, etc.
3. (The Big One) There is an innocent looking global variable config.
4. I think much of the codebase itself has added complexity because config doesn’t really abstract any of the implementation details, either. This line is an example - why does it need to know that changelog_sections is a comma-separated string? I am happy to discuss this in more depth - but I think the current implementation is limited by being only a simple str: str key-value store. Leveraging Python’s rich type system - lists, enums, namedtuples, etc - in my opinion will drastically simplify the code outside of the configuration setup.

The reason that the config variable (respectfully!) sucks to work with is it pops up absolutely everywhere, effectively as hidden parameters to every function. Take this function for example:

def current_commit_parser() -> Callable:
    """Get the currently-configured commit parser
    :raises ImproperConfigurationError: if ImportError or AttributeError is raised
    :returns: Commit parser
    """

    try:
        # All except the last part is the import path
        parts = config.get("commit_parser").split(".")
        module = ".".join(parts[:-1])
        # The final part is the name of the parse function
        return getattr(importlib.import_module(module), parts[-1])
    except (ImportError, AttributeError) as error:
        raise ImproperConfigurationError(f'Unable to import parser "{error}"')

What if config.get("changelog_components") is None? What is the default - I need to look it up, potentially in the docs (e.g. for commit_version_number, where the behaviour depends on other params). I know having worked through it that we ensure it’s populated through defaults.cfg, but now a change to defaults.cfg might ripple throughout the codebase in an at least semi-obscure manner.

When we talk about adding features like multibranch support, or bugs like versions not being set correctly, it’s hard to trace the source back to poor configuration.

Also when it comes to testing, there’s a lot of mocking and patching required (e.g. test_should_build) just to test different configuration options. These tests don’t measure the impact of unrelated parameter default values, and require care to make sure that all the right configuration variables are set and kept in sync with the source code.

The PR I’m working on will look to address just these 4 things - my goal is actually not to add any functionality nor fix any bugs, but to enable feature and bugfix development to happen much more smoothly. I intend to follow that PR up with another for multibranch support, which I guess would take python-semantic-release to 8.0, but addressing this will make testing-for - and fixing - the other bugs simpler.

The PR I’m working on is to address these three things only - the end goal is to have python-semantic-release completely unchanged in terms of features or bug fixes, and to have a like-for-like which is easier to test, fix and develop further.

Possible implementation

Describe any ideas you have for how this could work.

I’m writing my PR’s first commit to simply add parameters all over the codebase in all the functions, so that they can be passed in. I’m doing this irrespective of hacks, needed to hard-code into if/else statements, just to show the scale of changes to make this work (and so it doesn’t just look like a senseless rewrite).

Following that commit I will make 1+ more gradually cleaning up the mess of the first.

I appreciate that this might be a fairly significant change to the codebase - it certainly feels like it as I’m going through it! - but I do genuinely believe that it will be worth it, and that many other things will be easier to achieve once this is done.

Hoping to have the PR raised in the next couple of weeks 🎉

[bernardcooke53]

All the code is now at https://github.com/python-semantic-release/python-semantic-release/tree/8.0.x

[relekang]

I’ve created https://github.com/orgs/python-semantic-release and will configure and try to move over the next few days.