RedundantListenerException when updating to mockito 3.2.4 with parallel test execution

Hi!

I am trying to update a codebase to mockito 3.2.4!

After the update some tests fail with

org.mockito.exceptions.misusing.RedundantListenerException: 

Problems adding Mockito listener.
Listener of type 'MismatchReportingTestListener' has already been added and not removed.
It indicates that previous listener was not removed according to the API.
When you add a listener, don't forget to remove the listener afterwards:
  Mockito.framework().removeListener(myListener);
For more information, see the javadoc for RedundantListenerException class.

I was able to find out that the tests do not fail if I change the configuration of the maven-surefire-plugin to not run test in parallel.

The codebase uses maven-surefire-plugin 2.22.1 maven-compiler-plugin 3.8.0 junit 4.12 java 1.8

I created an example project which reproduces the problem https://github.com/julius-d/mockito-upgrade-problem clone it and run mvn test It will fail with RedundantListenerException The tests do nothing special, they only waste time:

  @Test
  public void name() {
    IntStream.range(0,  Integer.MAX_VALUE).boxed()
        .forEach(i -> assertEquals(i, i));
  }

• The mockito message in the stacktrace have useful information, but it didn’t help
• The problematic code (if that’s possible) is copied here; Note that some configuration are impossible to mock via Mockito
• Provide versions (mockito / jdk / os / any other relevant information)
• Provide a Short, Self Contained, Correct (Compilable), Example of the issue (same as any question on stackoverflow.com)
• Read the contributing guide

[mockitoguy]

I dug into this issue here’s what happens. When Maven Surefire plugin executes the tests with parallel “suitesAndClasses” feature I see an interesting symptom:

• one thread runs “withBefores” of the test (this is where the listener is created): https://github.com/mockito/mockito/blob/release/3.x/src/main/java/org/mockito/internal/runners/DefaultInternalRunner.java#L42
• different thread runs the actual test (this is where the listener is removed): https://github.com/mockito/mockito/blob/release/3.x/src/main/java/org/mockito/internal/runners/DefaultInternalRunner.java#L64

We expect the same thread to “start” the test and “finish” it. Internally, Mockito uses “ThreadLocal” to protect global state (global state is required to accommodate nice & sweet Mocking API).

What’s next?

• try to fix MockitoJUnitRunner. This is not something easy to reproduce let alone fix. More over, JUnit Runner is old - now we use JUnit5 (JUnit Jupiter). Mockito team won’t prioritize work on JUnit4 Runners. Take a stab at it if you feel strong! (I opened a PR that makes your repro repo a bit easier to work with: https://github.com/julius-d/mockito-upgrade-problem/pull/1).
• workarounds:
  • use JUnit Rule instead of the Runner: public @Rule MockitoRule mockito = MockitoJUnit.rule().strictness(Strictness.STRICT_STUBS);. Rules are simpler, the JUnit Rules API is much cleaner.
  • use JUnit5.

Thoughts?

[Gr33nbl00d]

I found out something very interesting: I had some classes using MockitoRunner others were not using it (No Runner Defined). The problem is gone when i let the other classes also use the MockitoRunner