TestNG swallows exceptions silently if @ClassRule is used
See original GitHub issueTestcase:
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mycompany</groupId>
<artifactId>JUnitTestcase</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.9.8</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<trimStackTrace>false</trimStackTrace>
</configuration>
</plugin>
</plugins>
</build>
</project>
MyTest.java
import org.junit.ClassRule;
import org.junit.Test;
import org.junit.rules.ExternalResource;
public class MyTest {
@ClassRule
public static ExternalResource resource = new ExternalResource() {
@Override
protected void before() throws Throwable {
throw new IllegalArgumentException("before");
}
@Override
protected void after() {
throw new IllegalArgumentException("after");
}
};
@Test
public void myTest() {
System.out.println("yay!");
}
}
If I disable TestNG in the pom file I get:
Running MyTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.064 sec <<< FAILURE! - in MyTest
MyTest Time elapsed: 0.063 sec <<< ERROR!
java.lang.IllegalArgumentException: before
at MyTest$1.before(MyTest.java:12)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:367)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:274)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:161)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
If I enable TestNG I get:
Running MyTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.69 sec - in MyTest
I am expecting to get similar output whether the TestNG plugin is enabled or not.
The popular Dropwizard framework relies on @ClassRule
so any minor configuration problem leads to unit tests silently failing.
Issue Analytics
- State:
- Created 7 years ago
- Comments:5
Top Results From Across the Web
TestNG swallows exception - java - Stack Overflow
In my method below when I get a IOException when I run it from TestNG (this is expected) but all I see is...
Read more >JUnit 5 User Guide
With the exception of @Test , these create a container in the test ... will be used; any additional declarations will be silently...
Read more >Swallowed Exceptions: The Silent Killer of Java Applications
In this post, we're looking at understanding the negative impact of swallowed exceptions and learning how to fix them.
Read more >The Great Transformation - INCT/PPED
The great transformation: the political and economic origins of our time / Karl ... to politics, religion, and social relations.10 Polanyi's use of...
Read more >News — JRuby.org
Empty rescue block returns exception object instead of nil. ... This improves integration with Java frameworks that use reflection to access object state....
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
For anyone running into this in the future, here is how I worked around the problem (hat tip @juherr):
The issue comes from https://github.com/cbeust/testng/blob/285cf3d9825eb4ed6f05a35c1626c989e5aece96/src/main/java/org/testng/junit/JUnit4TestRunner.java#L137
In case of an exception into a
@ClassRule
(but into a@BeforeClass
too),tr
isnull
because the description is not related to a test. TestNG has a specific listener for configuration methods but JUnit has not and I have not enough context information in the object. BTW, I need to find a way to manage it.From TestNG point-of-view, the output is:
From JUnit point-of-view, the output is:
@cbeust If you have an idea, I’m open 😃