Assert test coverage

Quoting @lukaseder from Twitter:

Is there a simple JUnit based tool to check for code coverage by a test class X for a given tested class Y? E.g. a rule specifying that if I add a method m matching some pattern P to Y without adding a test to X, X should fail.

Some thoughts:

1. During each test class’ execution, we need to gather coverage data. I’d prefer using a library for that but we could do it ourselves as well. Whichever way we get it done, it mustn’t impact performance if the feature isn’t used.
2. The gathered data could be placed into the store and then made available in handy AssertJ-esque assertion object.
3. Fittingly annotated methods that are executed at the after all extension point could get such an instance injected and use it to assert desired test coverage.

This could look as follows:

@Coverage(ProductionCode.class)
class MyTestCases

	// [...] regular @Test methods

	@CoverageTest
	void allMethodsTested(CoverageAssertions assertThat) {
		assertThat.allPublicMethodsCalled();
	}

}

Steps 2. and 3. should be straightforward, the only challenge (to me) is 1.

Any thoughts?

[verhas]

I spent the last few days of my vacation writing a classloader

https://github.com/verhas/impostor

that may help you to solve this case. You never know what you inspire.

Run your test loaded by this classloader and configure the class implementing the expencive equals method to be impersonated by an impostor class that throws an AssertionError in case equals() is involved.

This is not a JUnit feature, just a Q&D solution to your specific need.

You may also use AspectJ to run the test and get similar structure.

Unless I totally misunderstood something.

[lukaseder]

That last one kind of sounds like could be solved with a spy from Mockito?

Interesting idea. First caveat, this is modular code, and it’s not public, so I cannot access the T type unless I use reflection. Perhaps there’s a modular hack for this, but it seems that reflection / instrumentation is cleaner, here. Interesting idea nonetheless.

Also, this only works (if it works) if your set of tests is in a single class (which might be problematic).

In my case, I’d like the entire test suite to be run in this mode. Simple to do with an agent…