Feature request: Do not fail the Maven build for a failed test

When running npm test (for example) and if it fails, the whole Maven build will fail too. Would it be possible to make the plugin act differently for goals invoked during test and integration-test phases so that failures in those are only reported as test failures?

If you can provide some guidance on if/how this can be achieved, I’d gladly take on the task of implementing it.

[kaqqao]

While the feature has been added, the opposite behavior is now broken i.e. the builds now never seem to fail, even with maven.test.failure.ignore set to false.

[cliffmeyers]

@kaqqao @eirslett

I ran into this issue yesterday. The behavior does appear broken in 1.4 (failures during the test phase are always ignored). In 1.6 it works as expected. In these examples, I’ve introduced a failing test case that is run via gulp + gulp-jest.

Run a gulp build that finishes with exit code 1:

macbook-15:blueocean-dashboard cmeyers$ mvn clean install
...
[ERROR] Test Suites: 1 failed, 2 skipped, 26 passed, 27 of 29 total
[ERROR] Tests:       1 failed, 8 skipped, 149 passed, 158 total
[ERROR] Snapshots:   0 total
[ERROR] Time:        26.889s
[ERROR] Ran all test suites.
[INFO] [10:52:58] tests failed or errored!
[ERROR] 
[ERROR] npm ERR! Darwin 15.6.0
[ERROR] npm ERR! argv "/Users/cmeyers/Development/code/blueocean/blueocean-dashboard/node/node" "/Users/cmeyers/Development/code/blueocean/blueocean-dashboard/node/node_modules/npm/bin/npm-cli.js" "run" "mvntest"
[ERROR] npm ERR! node v6.4.0
[ERROR] npm ERR! npm  v3.10.3
[ERROR] npm ERR! code ELIFECYCLE
[ERROR] npm ERR! blueocean-dashboard@0.0.1 mvntest: `gulp test`
[ERROR] npm ERR! Exit status 1
[ERROR] npm ERR! 
[ERROR] npm ERR! Failed at the blueocean-dashboard@0.0.1 mvntest script 'gulp test'.
[ERROR] npm ERR! Make sure you have the latest version of node.js and npm installed.
[ERROR] npm ERR! If you do, this is most likely a problem with the blueocean-dashboard package,
[ERROR] npm ERR! not with npm itself.
[ERROR] npm ERR! Tell the author that this fails on your system:
[ERROR] npm ERR!     gulp test
[ERROR] npm ERR! You can get information on how to open an issue for this project with:
[ERROR] npm ERR!     npm bugs blueocean-dashboard
[ERROR] npm ERR! Or if that isn't available, you can get their info via:
[ERROR] npm ERR!     npm owner ls blueocean-dashboard
[ERROR] npm ERR! There is likely additional logging output above.
[ERROR] 
[ERROR] npm ERR! Please include the following file with any support request:
[ERROR] npm ERR!     /Users/cmeyers/Development/code/blueocean/blueocean-dashboard/npm-debug.log
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

Run again with the maven.test.failure.ignore property set:

mvn clean test -Dmaven.test.failure.ignore
...
[ERROR] Test Suites: 1 failed, 2 skipped, 26 passed, 27 of 29 total
[ERROR] Tests:       1 failed, 8 skipped, 149 passed, 158 total
[ERROR] Snapshots:   0 total
[ERROR] Time:        25.227s
[ERROR] Ran all test suites.
[INFO] [10:56:22] tests failed or errored!
[ERROR] 
[ERROR] npm ERR! Darwin 15.6.0
[ERROR] npm ERR! argv "/Users/cmeyers/Development/code/blueocean/blueocean-dashboard/node/node" "/Users/cmeyers/Development/code/blueocean/blueocean-dashboard/node/node_modules/npm/bin/npm-cli.js" "run" "mvntest"
[ERROR] npm ERR! node v6.4.0
[ERROR] npm ERR! npm  v3.10.3
[ERROR] npm ERR! code ELIFECYCLE
[ERROR] npm ERR! blueocean-dashboard@0.0.1 mvntest: `gulp test`
[ERROR] npm ERR! Exit status 1
[ERROR] npm ERR! 
[ERROR] npm ERR! Failed at the blueocean-dashboard@0.0.1 mvntest script 'gulp test'.
[ERROR] npm ERR! Make sure you have the latest version of node.js and npm installed.
[ERROR] npm ERR! If you do, this is most likely a problem with the blueocean-dashboard package,
[ERROR] npm ERR! not with npm itself.
[ERROR] npm ERR! Tell the author that this fails on your system:
[ERROR] npm ERR!     gulp test
[ERROR] npm ERR! You can get information on how to open an issue for this project with:
[ERROR] npm ERR!     npm bugs blueocean-dashboard
[ERROR] npm ERR! Or if that isn't available, you can get their info via:
[ERROR] npm ERR!     npm owner ls blueocean-dashboard
[ERROR] npm ERR! There is likely additional logging output above.
[ERROR] 
[ERROR] npm ERR! Please include the following file with any support request:
[ERROR] npm ERR!     /Users/cmeyers/Development/code/blueocean/blueocean-dashboard/npm-debug.log
[ERROR] There are test failures.
Failed to run task: 'npm run mvntest' failed.
com.github.eirslett.maven.plugins.frontend.lib.TaskRunnerException: 'npm run mvntest' failed.
	at com.github.eirslett.maven.plugins.frontend.lib.NodeTaskExecutor.execute(NodeTaskExecutor.java:63)
	at com.github.eirslett.maven.plugins.frontend.mojo.NpmMojo.execute(NpmMojo.java:62)
	at com.github.eirslett.maven.plugins.frontend.mojo.AbstractFrontendMojo.execute(AbstractFrontendMojo.java:94)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutionException: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
	at com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutor.execute(ProcessExecutor.java:82)
	at com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutor.executeAndRedirectOutput(ProcessExecutor.java:64)
	at com.github.eirslett.maven.plugins.frontend.lib.NodeExecutor.executeAndRedirectOutput(NodeExecutor.java:29)
	at com.github.eirslett.maven.plugins.frontend.lib.NodeTaskExecutor.execute(NodeTaskExecutor.java:58)
	... 24 more
Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
	at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
	at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
	at com.github.eirslett.maven.plugins.frontend.lib.ProcessExecutor.execute(ProcessExecutor.java:74)
	... 27 more
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Also thanks to the contributor for helpful output like this:

[INFO] --- frontend-maven-plugin:1.6:npm (npm install) @ blueocean-dashboard ---
[INFO] testFailureIgnore property is ignored in non test phases
[INFO] Running 'npm install' in /Users/cmeyers/Development/code/blueocean/blueocean-dashboard

Two small suggestions that I could open a PR for if you’re willing @eirslett

• Perhaps we should add a similar logging statement during test phases that indicate the value of testFailureIgnore and whether failures will result in a broken build, and
• The stack trace is logged when test failures are ignored, but not in the other case (see first snippet). Should it be logged in both cases?