Restructure build matrix

Our current build configuration operates on the following sets:

• operating systems OS = { Ubuntu, Windows, macOs }
• Java versions J = { 8, 11, 16 }
• modularity M = { true, false }
• JUnit versions U = { 5.7.0, 5.7.1, 5.8.0-M1 }

But we don’t build all possible combinations OS x J x M x U, which would be 54 builds and slightly above the 20 builds GitHub runs in parallel for free. Even if we upgrade to team, which raises the limit to 60 (and costs 4$/member/month), we’d be just one additional Java or JUnit version away from going overboard.

Instead we build:

• OS x J x { false } x { 5.7.1 } in main-build.yml (9 builds)
• OS x { 11, 16 } x { true } x { 5.7.1 } in module-build.yml (6 builds)
• { Ubuntu } x { 8 } x { false } x U in junit5-build.yml (3 builds)

I find that combination of builds somewhat arbitrary. Also, just one more Java or JUnit version and we blow the limit of 20 builds and need to wait twice as long (worst case) for the entire set to finish.

Can we find a better way to create builds?

[aepfli]

@nipafx @Bukama @beatngu13 what do you think about such an approach? good bad ugly?

[aepfli]

I would also evaluate what do we really need on a merge request and what can be handled as an action on a merge into develop.

Generally speaking it is nice, if everything runs during a merge request - but is it really needed or is it needed all the time?

For verification if the code works (which imho the CI should guarantee) a small build is enough (to be honest we need it to get reminded of spotless).

What if we define two sets of CI jobs - the ones which run all the time, short fast, small set - and the ones we need to run before merging. It will not block a contributor and the contributor will get fast feedback. On the other hand, we as maintainer could eg. label or comment on the MR that we are ready to merge which could kickoff a bunch of more tests. Those builds are normally not timely critical, therefore we could even have more builds and exceed the limit.

This is just a suggestion and some thoughts about this. eg. GitLab is doing this for their Repositories too.