Current state of tests is not great, let's fix them

As the title says current state of tests is not where it should be especially considering the purpose of the mutmut itself is to improve unit tests. I feel that we need to address these issues sooner rather than later since it will make maintenance much easier in the future.

1. Current tests are more integration / end-to-end tests than unit tests
2. Big chunk of code is not properly unit tested
3. mutmut is not used on mutmut

Current tests are more integration / end-to-end tests than unit tests Here is the article by Michael Feathers that pretty much explains this point. Each unit test should test only one unit of the code, also no spinning disks for unit tests and no IO at all. Some parts of the code will, for sure, require refactoring to make them testable. There is a lot of IO in functions that should be moved up a level to make the code easy to test. Here is a great talk by Brandon Rhodes on how to “hoist IO”.

Big chunk of code is not properly unit tested It is hard to determine actual unit test coverage since coverage is calculated while running all tests. Currently it looks good, but there is a decent amount of edge cases that are not tested because it is hard to reproduce edge cases by writing integration tests. Coverage should be calculated when running only true unit tests.

mutmut is not used on mutmut This is the best way to make our unit tests better, that’s what mutmut does. Also this would make it easier for us to spot new use cases and potential improvements. But first, things need to be refactored.

Steps to mitigate:

• Making a clear separation between unit tests and integration / end-to-end tests. (different dirs/files and pytestmark)
• Stop merging pull requests that aren’t properly covered with unit tests.
• Chunk by chunk refactor code to be testable and add appropriate unit tests for it.
• After the old code is refactored, switch coverage to work only with unit tests.
• Start using mutmut on unit tests

Good thing is that code base is not that big, it is a great time to do something like this. Also current tests are good for what they do, we just need to add proper unit tests.

Please let me know what you think. 😃

[boxed]

I think we should cross that bridge when we get to it. This is a case of YAGNI. We should worry first about having solid mutations built in, and making mutmut fast and easy to use and generally improving the workflow with dealing with findings.

[boxed]

I do try to avoid classes at all costs. I agree that maybe I’ve gone too far here. What do you think about this @s-kovacevic ?