`JVMHookResourceReaper` fails in Quarkus continuous testing for multi module project

When running Quarkus continuous testing in a multi-module project, something like the following is logged when terminating via ctrl+c:

Exception in thread "Thread-368" com.github.dockerjava.api.exception.ConflictException: Status 409: {"message":"a prune operation is already running"}

        at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:245)
        at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.post(DefaultInvocationBuilder.java:124)
        at org.testcontainers.shaded.com.github.dockerjava.core.exec.PruneCmdExec.execute(PruneCmdExec.java:33)
        at org.testcontainers.shaded.com.github.dockerjava.core.exec.PruneCmdExec.execute(PruneCmdExec.java:13)
        at org.testcontainers.shaded.com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21)
        at org.testcontainers.shaded.com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35)
        at org.testcontainers.shaded.com.github.dockerjava.core.command.PruneCmdImpl.exec(PruneCmdImpl.java:100)
        at org.testcontainers.utility.JVMHookResourceReaper.prune(JVMHookResourceReaper.java:53)
        at org.testcontainers.utility.JVMHookResourceReaper.lambda$performCleanup$3(JVMHookResourceReaper.java:28)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
        at org.testcontainers.utility.JVMHookResourceReaper.performCleanup(JVMHookResourceReaper.java:28)
        at java.base/java.lang.Thread.run(Thread.java:833)
Exception in thread "Thread-76" com.github.dockerjava.api.exception.ConflictException: Status 409: {"message":"a prune operation is already running"}

        at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:245)
        at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.post(DefaultInvocationBuilder.java:124)
        at org.testcontainers.shaded.com.github.dockerjava.core.exec.PruneCmdExec.execute(PruneCmdExec.java:33)
        at org.testcontainers.shaded.com.github.dockerjava.core.exec.PruneCmdExec.execute(PruneCmdExec.java:13)
        at org.testcontainers.shaded.com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21)
        at org.testcontainers.shaded.com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35)
        at org.testcontainers.shaded.com.github.dockerjava.core.command.PruneCmdImpl.exec(PruneCmdImpl.java:100)
        at org.testcontainers.utility.JVMHookResourceReaper.prune(JVMHookResourceReaper.java:53)
        at org.testcontainers.utility.JVMHookResourceReaper.lambda$performCleanup$3(JVMHookResourceReaper.java:28)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
        at org.testcontainers.utility.JVMHookResourceReaper.performCleanup(JVMHookResourceReaper.java:28)
        at java.base/java.lang.Thread.run(Thread.java:833)
Exception in thread "Thread-113" com.github.dockerjava.api.exception.ConflictException: Status 409: {"message":"a prune operation is already running"}

        at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:245)
        at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.post(DefaultInvocationBuilder.java:124)
        at org.testcontainers.shaded.com.github.dockerjava.core.exec.PruneCmdExec.execute(PruneCmdExec.java:33)
        at org.testcontainers.shaded.com.github.dockerjava.core.exec.PruneCmdExec.execute(PruneCmdExec.java:13)
        at org.testcontainers.shaded.com.github.dockerjava.core.exec.AbstrSyncDockerCmdExec.exec(AbstrSyncDockerCmdExec.java:21)
        at org.testcontainers.shaded.com.github.dockerjava.core.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:35)
        at org.testcontainers.shaded.com.github.dockerjava.core.command.PruneCmdImpl.exec(PruneCmdImpl.java:100)
        at org.testcontainers.utility.JVMHookResourceReaper.prune(JVMHookResourceReaper.java:53)
        at org.testcontainers.utility.JVMHookResourceReaper.lambda$performCleanup$3(JVMHookResourceReaper.java:28)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
        at org.testcontainers.utility.JVMHookResourceReaper.performCleanup(JVMHookResourceReaper.java:28)
        at java.base/java.lang.Thread.run(Thread.java:833)

I’m short on time currently, but I know the internals a bit and I suppose this is due to testcontainers (org.testcontainers.utility.ResourceReaper.DEATH_NOTE respectively) being loaded multiple times by mutliple classloaders. So each shutdown hook might be thinking it’s alone in the current JVM, but it isn’t, hence all run concurrently.

This of course only happens if TESTCONTAINERS_RYUK_DISABLED is true, but I want TC to be as fast as possible to provide a reusable(!) MariaDB container and I found the RYUK setup to waste a measurable amount of time. I don’t need/want any kind of cleanup by TC.

I see two options:

• provide a env/system property to opt out of the shutdown hook(s)
• and/or possibly don’t register any cleanup in case all containers involved are reusable/reused

[famod]

Ok, so this isssue here will only affect people that have non-reusable containers (or just don’t have resuse activated?) and disable ryuk via TESTCONTAINERS_RYUK_DISABLED. Not having reuse enabled is probably very common, but having disabled ryuk is probably rare (?).

In my case, removing TESTCONTAINERS_RYUK_DISABLED is the way to go! 👍

[kiview]

@famod You are right, that is the intention behind #4938, to provide better startup performance in those cases.