Please disable `UnnecessaryStubbingException` by default

I recently upgraded from 1.x to 2.2.8 and saw hundreds of tests breaking due to UnnecessaryStubbingException. While I understand the motivation and certainly agree that having unused stubs can be a code smell in some cases, I believe that failing tests due to it is too strict.

I will try to motivate why through some use cases I have encountered trying to fix my (now) failing tests.

Motivational Example 1

One of the key features of a good unit test is that it is robust. When faced with a complex task there may be several possible ways to implement the task, some of which interact with dependants in different ways. Or even interact with different (sub) dependants. This refactoring is one of the core tennets in test driven development.

Consider the following code:

int foo(){
    int x = dependant.getX();
    return doComplexComputationUsingOtherDependantsThatReturns0IfXEquals0();
}

Then write a lot of tests with all the necessary stubs for the computation and test foo. Later on some one refactors foo for performance reasons, turns out 0 is a very common value of x:

int foo(){
    int x = dependant.getX();
    if(x == 0)
        return 0; // early exit
    return doComplexComputationUsingOtherDependantsThatReturns0IfXEquals0();
}

Now test cases that test for x=0 will no longer interact with some dependants and the test will fail with UnnecessaryStubbingException even though the tests are correct. The tests have become brittle.

Motivational Example 2

Consider a complex model of a Car, the car has a DriveTrain which has an Engine and the engine has many SparkPlugs. In a similar way the Car has a Chassis which has 2 or more Doors etc. You get the picture, a complex model.

Now consider some computational classes like MaxSpeedCalculator, PowerToWeightRatioCalculator each which takes a Car and uses different portions of it for their computations.

When testing these calculator classes, you end up having to mock the basic structure of a Car every time which results in a lot of boiler plate. One way of solving this is to introduce a MockCar class which is basically contains mock objects of all the relevant classes and has basic stubs setup. For example:

public class MockCar{
     final public Car car = mock(Car.class);
     final public DriveTrain driveTrain = mock(DriveTrain.class);
     final public Engine engine = mock(Engine.class);

     public MockCar(){
         when(car.getDriveTrain()).thenReturn(driveTrain); 
         when(driveTrain.getEngine()).thenReturn(engine);
    }

and so forth. This removes a lot of boiler plate and is in no way dangerous. I would argue that this is good.

However the UnnecessaryStubbingException prevents re-use and boilerplate reduction through this technique.

In summary

This change makes it hard or impossible to write robust tests where complex relations between objects exist. It also prevents some code duplication reduction techniques.

It is also good practice to test for behaviour and not implementation (black box testing for example) which is also made harder through the above.

[LiSongMWO]

Clarification to Example 1

Good tests should be robust; they should only fail if the code under test contains a flaw. A test that fails after a correct refactoring is a maintenance burden and a waste of developer time.

Consider the following (contrieved) SSCCE: https://gist.github.com/EmilyBjoerk/cff3b7cb60b9f1d903a6ca8ea5b7d0c9

Un/Re-commenting the marked line will break the tests every time even though the code under test is correct with and without the commented line. Clearly these tests are brittle, and this brittleness is enforced by UnnecessaryStubbingException. This goes against good testing practices.

I believe that a more flexible line of thought regarding stubs is better for maintainable tests: “This method is allowed to use these methods on these dependants and they will return expected values, but the implementation is not required to do so. However the answer returned must be X regardless.”. This is what separates a mock from a stub, whether the code under test is required to or/and allowed to call certain methods. The UnnecessaryStubbingException is implicitly turning each stub into a mock.

Another way to look at it, the existence of UnnecessaryStubbingException is an implicit verify on each when, and it is a code smell to have excessive verification of results that are not specific to the given test. In other words, UnnecessaryStubbingException in itself automatically introduces an implicit code smell in each and every test.

On Example 2

Why do you consider such mock structures (note that they only mock the structure so that you can reach other (default) mocks easier) to be a code smell? What ill effects can reasonably be expected to occur from this?

That said I agree, I’m not happy about MockLoadoutContainer.java but the alternative is thousands of lines of setup code spread among many many tests that is literally 1 or two lines away from copy paste. And copy paste is an even bigger code smell than MockLoadoutContainer.java.

What I really want is to be able to do:

Loadout l = mock(Loadout.class);
when(l.getComponent(Location.RightArm).getInternalComponent().getFreeSlots()).thenReturn(3);

and have it auto create and keep track of the mocks for Component and InternalComponent.

However this is not possible with Mockito but MockLoadoutController makes it possible for the main model in my application:

MockLoadoutContainer mlc = new MockLoadoutContainer();
when(mlc.loadout.getComponent(Location.RightArm)
                .getInternalComponent().getFreeSlots()).thenReturn(3);
// or equivalently
when(mlc.ira.getFreeSlots()).thenReturn(3);

because the relations between all components have already been setup, some of them might not be used but that’s a small price to avoid massive code duplication. Also I have not had a single case where inadvertent use of any of the mocks in MockLoadoutContainer was the cause of a flaw in the code. Without MockLoadoutContainer I’m forced to write

InternalComponent ira = mock(InternalComponent.class);
Component cra = mock(Component.class);
Loadout loadout = mock(Loadout.class);
when(loadout.getComponent(Location.RightArm)).thenReturn(cra);
when(component.getInternalComponent()).thenReturn(ira);
when(ira.getFreeSlots()).thenReturn(3);

Which is a factor 3 code expansion which is essentially copy pasted all over just to setup one component. Usually I use all 8 components (Right/Left-Arm/Leg/Torso, Center Torso and Head) in the code under test you see how this code duplication quickly amounts to around 4*8=32-ish lines per test for the numerous classes that use all the components which range in the 30-ish (32 lines * 30 tests = 960 lines). This makes writing tests a hassle and is a serious maintenance burden.

I’m would be really happy to change my code if you have a good suggestion that does not trigger UnnecessaryStubbingException and does not involve copy paste and code duplication. I’m in no way set in stone on this implementation. If I can reduce my code while keeping the same behaviour I’m all for it!

P.S. I found the silent runner a while after posting the first message and I’m now using it, please keep it around.

P.S.S. As a stop gap, please improve the JavaDoc of UnnecessaryStubbingException to clearly mention MockitoJUnitRunner.Silent and the exact cases where it will be triggered.

[mockitoguy]

Thank you very much for feedback and taking the time to describe your use cases so clearly!

To make it clear, your ask is to make the ‘unncessary stubbing’ exception emitted by JUnitRunner optional, and turned off by default.

Motivational Example 1

I understand this example as ‘default stubbing’. You have a stubbing that is applicable to most test methods, but not all in give test class, you put this stubbing in setup() or test constructor. This scenario is supported by JUnitRunner - if the stubbing is used in at least one test method, no exception is emitted.

Please elaborate this example because at the moment I don’t know why the new behavior is problematic for this example.

Motivational Example 2

I consider such elaborate stubs, implemented with Mockito a code smell. It’s classic overmocking: hundreds of interactions are stubbed, many, many mocks created. For such data structures, “hand stubs” work much better: cleaner code, easier to understand.

I would not like to have class like MockLoadoutContainer.java in my codebase.

We can continue discussing if this is indeed a code smell (and please do state your arguments for) and I can predict the debate will go on and on. Please accept that in order to change the default, we need to hear from more users and have more compelling use cases.

You are welcome to use MockitoJUnitRunner.Silent or MockitoJUnit.rule() (the rule does not throw unnecessary stubbing exception - it only prints the warning to the console). Using those APIs you can hold on to your style of writing tests / building stubs. Mockito is opinionated but it is not dogmatic.

Thanks again for great feedback and looking forward to your reply!