IllegalStateException when using JUnit 5.3's support for parallel test execution

JUnit 5.3 M1 allows for a parallel execution of tests (if you add junit.jupiter.execution.parallel.enabled = true to src/test/resources/junit-platform.properties)

Using it with spring-restdocs and multiple tests triggers this error: Context already exists. Did you forget to call afterTest()?

Stacktrace

java.lang.IllegalStateException: Context already exists. Did you forget to call afterTest()?
	at org.springframework.restdocs.ManualRestDocumentation.beforeTest(ManualRestDocumentation.java:72)
	at org.springframework.restdocs.RestDocumentationExtension.beforeEach(RestDocumentationExtension.java:45)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeEachCallbacks$0(TestMethodTestDescriptor.java:129)
	at org.junit.jupiter.engine.execution.ThrowableCollector.execute(ThrowableCollector.java:40)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:155)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeEachCallbacks(TestMethodTestDescriptor.java:128)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:107)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:58)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:134)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:128)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:109)
	at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.compute(ForkJoinPoolHierarchicalTestExecutorService.java:161)
	at java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool.helpStealer(ForkJoinPool.java:1958)
	at java.util.concurrent.ForkJoinPool.awaitJoin(ForkJoinPool.java:2047)
	at java.util.concurrent.ForkJoinTask.doJoin(ForkJoinTask.java:390)
	at java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:719)
	at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService.joinConcurrentTasksInReverseOrderToEnableWorkStealing(ForkJoinPoolHierarchicalTestExecutorService.java:138)
	at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService.invokeAll(ForkJoinPoolHierarchicalTestExecutorService.java:112)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:128)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:109)
	at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.compute(ForkJoinPoolHierarchicalTestExecutorService.java:161)
	at java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

EDIT: it’s also entirely possible this is the bug on JUnit’s side, since it seems you call the context-creating and removing methods before and after each test, which seems OK 😄

[marcphilipp]

otherwise extensions cannot, easily, be stateful.

That’s exactly right. At least they should not use mutable/stateful instance variables like RestDocumentationExtension currently does:

https://github.com/spring-projects/spring-restdocs/blob/aedb9a7babe5ac8b9eb0b0bacc0909fe1b6e3abe/spring-restdocs-core/src/main/java/org/springframework/restdocs/RestDocumentationExtension.java#L34-L51

Instead, the Jupiter API provides a separate concept to store extension state: ExtensionContext.getStore(Namespace). This way JUnit takes care of using the correct store for the current test and the extension does not have to worry about different tests that might even run in parallel.

Here’s how that would look for RestDocumentationExtension:

public class RestDocumentationExtension implements Extension, BeforeEachCallback,
		AfterEachCallback, RestDocumentationContextProvider, ParameterResolver {

	@Override
	public void beforeEach(ExtensionContext context) throws Exception {
		Class<?> testClass = context.getRequiredTestClass();
		Method testMethod = context.getRequiredTestMethod();
		getDelegate(context).beforeTest(testClass, testMethod.getName());
	}

	@Override
	public void afterEach(ExtensionContext context) throws Exception {
		getDelegate(context).afterTest();
	}

	private ManualRestDocumentation getDelegate(ExtensionContext context) {
		var namespace = Namespace.create(RestDocumentationExtension.class, context.getUniqueId());
		return context.getStore(namespace).getOrComputeIfAbsent(ManualRestDocumentation.class);
	}

	// ...
}

@wilkinsona Please let me know if I should submit a PR.

[wilkinsona]

Thanks for the input, @sbrannen and @marcphilipp.