gradle reliability roadmap

Reliable builds during development with support for combined gradle & IntelliJ IDEA.

I have a sort of roadmap for this. Maybe it’s the sort of thing that should be a Milestone or have a Project board, but I’ll start by writing here so it’s someplace besides my head:

• [regression] #4014: transient modules/*.jar
• docs: remove mentions of gradlew idea and iml files from wiki:Developing Modules
• determine whether gradlew idea and gradlew cleanIdea do harmful things and if we need to disable them
• gradle terasology-modules plugin. started by #3993.
  • extract cacheReflections task. work in progress, #4022. also relates to #2445.
  • make terasology-module plugin not depend on /config/gradle/publish.gradle. Surprisingly enough, it seems to work, but it is unexpected source code organization and a blocker to the plugin being used outside this repository. https://github.com/MovingBlocks/Terasology/blob/957543938f28d703e3175e0ca6f2c5f64da0dd1d/buildSrc/src/main/kotlin/terasology-module.gradle.kts#L89
• #4021: reflections and other work with resources will be easier if engine gets rid of its dev source set, see https://github.com/MovingBlocks/Terasology/issues/3979#issuecomment-633094908
• make syncAssets task (or copyResourcesToClasses, as it’s called in engine’s build) gradle-safe. or make unnecessary by getting Gestalt to allow us to register more than one filesystem-directory to a gestalt-module. see #4555, #3957 & https://github.com/MovingBlocks/Terasology/blob/957543938f28d703e3175e0ca6f2c5f64da0dd1d/engine/build.gradle#L277-L278
• remove outputDir overrides on IDEA config; colliding gradle and IDEA build directories is an anti-goal.
  • in modules: https://github.com/MovingBlocks/Terasology/blob/957543938f28d703e3175e0ca6f2c5f64da0dd1d/buildSrc/src/main/kotlin/terasology-module.gradle.kts#L235-L236
  • in engine: https://github.com/MovingBlocks/Terasology/blob/957543938f28d703e3175e0ca6f2c5f64da0dd1d/engine/build.gradle#L301-L302
  • in facade: https://github.com/MovingBlocks/Terasology/blob/957543938f28d703e3175e0ca6f2c5f64da0dd1d/facades/PC/build.gradle#L426-L427
• decouple facade’s configuration from engine and module projects. Goal is to improve confidence that engine and all modules will be up-to-date whenever facade’s exec targets are launched. Hope is that following multi-project best practices makes this more reliable. See also Decoupled Projects.
• #4245: streamline facade’s exec tasks. There is a lot of duplication between them. They can probably also take advantage of the java-application run task.
• IntelliJ mystery: no longer able to effectively rebuild individual modules? Right click module -> Rebuild (if no previous build directory) may result in nothing building, yet Build -> Build Project builds it fine (item added and observed by Cerv)

Additional Criteria

• Use task configuration avoidance and lazy configuration to keep big multi-project workspaces fast.
• Retain compatibility with run-time reloading of classes (with IDE) and assets (with gestalt).

Optional Side Quests

• separate module test-dependencies from their application dependencies.
• publish terasology gradle plugin(s) for use from other repositories.
• extractNatives - is there a better place for this to be so IntelliJ will recognize it as a dependency and always make sure the libs exist before it runs the facade? does anything else need them other than facades? (e.g. tests, compile-time dependencies)
  • progress toward this in #4347

What you were trying to do

just, like, be consistently able to build and run stuff

What actually happened

I learned more about gradle than I ever wanted to know.

[keturn]

decouple facade’s configuration from engine and module projects. Goal is to improve confidence that engine and all modules will be up-to-date whenever facade’s exec targets are launched. Hope is that following multi-project best practices makes this more reliable.

After #4454, this is in a much better state with regards to how facade & module are organized according to gradle’s recommendations.

IntelliJ mystery: no longer able to effectively rebuild individual modules? Right click module -> Rebuild (if no previous build directory) may result in nothing building, yet Build -> Build Project builds it fine (item added and observed by Cerv)

Time to re-test this after all the recent configuration changes.

make syncAssets task (or copyResourcesToClasses, as it’s called in engine’s build) gradle-safe. or make unnecessary by getting Gestalt to allow us to register more than one filesystem-directory to a gestalt-module.

The upgrade to v7 for gestalt-module and gestalt-asset is on the near-term roadmap now, after which I’d be interested in taking another look at this from the module-loading side.

extract cacheReflections task.

There are persistent rumors about some work in progress that’s going to replace our use of Reflections. Not sure of what forum or issue I can reference for the status of that. It might be https://github.com/MovingBlocks/gestalt/pull/83 ? But there’s no description on that so I’m not sure if it’s the right thing, or only part of the thing, or what.

#4022 got stickier than expected, and there’s been no news on the gradle issues it links. (Which doesn’t mean nothing has changed in gradle, but if it has nobody realized the change is relevant for those old issues and updated them.) What I’m saying is I’d be happy to have an excuse to drop #4022, but I don’t really know if whatever is replacing it is any easier.

[keturn]

FYI: Gradle 6.7 has entered release candidate stage, which usually indicates it will be stable within a week or two.

I haven’t seen anything between 6.4 – 6.6 that makes upgrading a priority for us, but I will be interested in grabbing 6.7 before resuming work on the terasology-module gradle plugin in /buildSrc/. We don’t exactly depend on the features released in 6.7 but they did a significant amount of work under the hood to support the new Java Toolchain stuff, and that will be relevant for the parts of the build that get a little too involved with the java tasks’s implementation details, i.e. #4022.

I don’t see anything I expect to give us trouble in the upgrade guide.