Modernizing gitlint's build and test tooling: discussion

I’d like to get some input on best-practices around python build/test/project management/CI tooling. I want to streamline the gitlint developer and release experience to:

1. Reduce release overhead (eventually: more frequent and smaller releases)
2. Clean up some unused and outdated code (most notably in ./run_tests.sh)
3. Get gitlint up-to-date with recent python test, build and deploy best-practices (and personally learn a few things along the way)

Background

Test running

• Today gitlint uses a custom shell script run_tests.sh which wraps most other test tools like pytest, pylint, black, coverage, and radon, as well as a build (python setup.py sdist bdist_wheel) test.
• run_tests.sh also has some other utility functions like clean and some outdated python and docker environment management related code that is not used anymore.
• I’d like to get rid of run_tests.sh entirely and move to a python tool or script.
• I’d like this tool to do python environment management (like tox), so it becomes easier to run these tools against multiple python environments locally. Currently we use a test matrix in CI, perhaps this can be (partially) handled by the testing tool instead (so it’s independent of GHA).

Builds

• gitlint consists of 2 python packages gitlint (root dir) and gitlint-core (gitlint-core dir), whatever tool/script we use should be able to support multiple python packages in a single repo / root dir.
• I’d like to move away from setup.py towards declarative config and pyproject.toml specifically. The build system should be PEP517 compliant.
• I’d like to remove the various requirements*.txt files in favor of using extra_requires for test, docs, etc.
• A publish command for publishing to PyPI is a nice to have.
• An immediate goal is to have a fully automated CI pipeline that publishes to Test PyPi on every commit. Other than writing the required pipeline that does this, this also requires generating a unique version for every commit - I assume that would probably be based on the git hash of main (seems to be the norm in my experience). Sample version string: 0.19.dev-abc123. I know some build systems or plugins help with this type of version management - that would be a nice to have. Currently we maintain gitlints version manually in 2 files: setup.py and gitlint-core/gitlint/__init__.py (also used to display the version in the UI).

Candidates

• I really like the idea of having a single project management tool that handles/wraps both the build and script running parts. This keeps the learning curve low and the developer experience good.
• I’ve looked at pyinvoke before, but that was 5y ago. I don’t think pyinvoke is still a candidate, too much manual work and too customized - but it might be.
• I have experience using poetry and like it, but they’ve also done some weird things - I know opinions are divided on it. I think @asottile put it very eloquently in his video. So I think poetry is out as well.
• I know there’s a ton of other tools, I’ve been looking at a few of them: https://packaging.python.org/en/latest/key_projects/
• Specific call-outs to pip-tools, pre-commit and tox which I think are all in the running but none of them solve the problem entirely (so some “glue” would be needed).
• I noticed that mkdocs recently adopted hatch and that got me very intrigued. Right now that’s what I’m mostly leaning towards (at least to give it a try).

I’m planning to work on this in the next few weeks and would appreciate input on it. 🙏

Final Note: There’s many roads that lead to Rome, I think it’s likely there won’t be any consensus on this, in which case I’ll make a judgement call and just try something 😃

CC: @sigmavirus24, @asottile, @webknjaz, @ssbarnea, @andersk

[ofek]

Hatch maintainer here! Let me know if I can help

[jorisroovers]

Thanks for all your inputs - definitely insightful. Really appreciated!

I’m going to give hatch a shot, I’ve implemented the basic setup in #384. I really like hatch 🫶, the end result is very elegant IMO. I’ll leave the PR open for a few days to gather input before merging it.

I haven’t implemented the CI deployment too Test PyPi on every commit yet, but I can see a few ways to do that (potentially using https://github.com/ofek/hatch-vcs) - I’ll do that next.