Exceptions and Errors ignored due to unchecked Futures
See original GitHub issueDescribe the bug
@keith-turner reported an issue where he had the wrong accumulo-core
jar file deployed to a cluster and the TabletServer continued to run. When he changed the code to catch Throwable
instead of catch Exception
, this stacktrace was captured:
java.lang.NoSuchFieldError: TSERV_WAL_BUFFER
at org.apache.accumulo.tserver.log.DfsLogger.open(DfsLogger.java:425) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at org.apache.accumulo.tserver.log.TabletServerLogger$2.run(TabletServerLogger.java:277) ~[accumulo-tserver-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
at org.apache.accumulo.core.trace.TraceWrappedRunnable.run(TraceWrappedRunnable.java:52) ~[accumulo-core-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
at java.lang.Thread.run(Thread.java:829) [?:?]
It turns out that the issue here is that ExecutorService.submit
was being called and the Future
result object was not being checked. When he changed the code to use ExecutorService.execute
instead, then the AccumuloUncaughtExceptionHandler
was invoked, the Error printed in the log, and the VM halted. When using submit
, the TabletServer remained up, but in a Zombie non-functional state.
It appears that Futures are capturing Exceptions and Errors, so the UncaughtExceptionHandler is not being invoked. It would be nice if SpotBugs had a rule for this. The closest I found is https://github.com/spotbugs/spotbugs/issues/950, but it’s still open.
Issue Analytics
- State:
- Created 2 years ago
- Comments:12 (12 by maintainers)
I added the following to the
errorprone
profile in the parent pomIt found 16 errors in the tserver module when I ran
mvn -pl server/tserver -Perrorprone clean package -DskipTests
Removing the DisableAllWarnings is fine, but ErrorProne is fairly aggressive so there will be a lot of SuppressWarning annotations to add. Of course once all warnings are parsed you can always update the pom to ignore warnings we do not care about.