Test runner that sets System.err causes Bloop server to become unresponsive

Using a native-image of bloopgun, I’m unable to run bloop exit

❯ bloop exit
error: Unexpected error forces client exit!
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:397)
	at bloop.shaded.snailgun.protocol.Protocol.$anonfun$processChunkFromServer$1(Protocol.scala:181)
	at scala.util.Try$.apply(Try.scala:213)
	at bloop.shaded.snailgun.protocol.Protocol.processChunkFromServer(Protocol.scala:180)
	at bloop.shaded.snailgun.protocol.Protocol.sendCommand(Protocol.scala:108)
	at bloop.shaded.snailgun.TcpClient.run(TcpClient.scala:34)
	at bloop.bloopgun.BloopgunCli.executeCmd$1(Bloopgun.scala:255)
	at bloop.bloopgun.BloopgunCli.fireCommand(Bloopgun.scala:261)
	at bloop.bloopgun.BloopgunCli.run(Bloopgun.scala:216)
	at bloop.bloopgun.Bloopgun$.main(Bloopgun.scala:560)
	at bloop.bloopgun.Bloopgun.main(Bloopgun.scala)

After running this command once, I’m unable to communicate with the Bloop server with other commands

❯ bloop help
error: Unexpected error forces client exit!
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:397)
	at bloop.shaded.snailgun.protocol.Protocol.$anonfun$processChunkFromServer$1(Protocol.scala:181)
	at scala.util.Try$.apply(Try.scala:213)
	at bloop.shaded.snailgun.protocol.Protocol.processChunkFromServer(Protocol.scala:180)
	at bloop.shaded.snailgun.protocol.Protocol.sendCommand(Protocol.scala:108)
	at bloop.shaded.snailgun.TcpClient.run(TcpClient.scala:34)
	at bloop.bloopgun.BloopgunCli.executeCmd$1(Bloopgun.scala:255)
	at bloop.bloopgun.BloopgunCli.fireCommand(Bloopgun.scala:261)
	at bloop.bloopgun.BloopgunCli.run(Bloopgun.scala:216)
	at bloop.bloopgun.Bloopgun$.main(Bloopgun.scala:560)
	at bloop.bloopgun.Bloopgun.main(Bloopgun.scala)

[olafurpg]

I may have tracked down the root cause by running bloop console PROJECT_NAME

bloop console <redacted>
Loading...
Compiling (synthetic)/ammonite/predef/interpBridge.sc
scala.reflect.internal.FatalError: Error accessing /Users/lgeirsson/workspace/source/.pants.d/export-dep-as-jar/export-dep-as-jar/252d64521cf9/tmpn0s2b0vm.jar
  scala.tools.nsc.classpath.AggregateClassPath.$anonfun$list$3(AggregateClassPath.scala:99)

This Bloop JSON file contains jars on the classpath, which don’t exist. This JSON file was created with an older version of the Pants exporter. I am unable to reproduce when using the latest version of the Pants exporter (which copies over jars and uses an immutable classpath).

I believe Bloop should still be able to gracefully handle this situation, maybe by reporting a warning under --verbose when the classpath contains non-existing entries.

[jvican]

Thanks for the follow-up and the reproduction, I’m happy we tracked this issue down!

Given that System.setErr could be triggered by classloading the test jars, I believe we should add some logic to bloop to reset back streams right after classloading is done. Nailgun needs the streams to be instances of their concrete thread-local stream classes so that it can redirect in, out and err directly to the socket streams of every client, even when several clients run at the same time.

The implementation of the fix on the bloop side should account for the fact that test classloading can happen concurrently when parallel execution is enabled.