first run of Scala 3 compiler sometimes fails

Output on OSX revealed a stack trace:

java.util.concurrent.ExecutionException: org.eclipse.lsp4j.jsonrpc.ResponseErrorException: expected whitespace or eof got u (line 1, column 2316)
    at org.eclipse.lsp4j.jsonrpc.RemoteEndpoint.handleResponse(RemoteEndpoint.java:209)
    at org.eclipse.lsp4j.jsonrpc.RemoteEndpoint.consume(RemoteEndpoint.java:193)
    at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.handleMessage(StreamMessageProducer.java:192)
    at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.listen(StreamMessageProducer.java:94)
    at org.eclipse.lsp4j.jsonrpc.json.ConcurrentMessageProcessor.run(ConcurrentMessageProcessor.java:99)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
...

which suggests that sometimes there is additional output being sent after some JSON. Given that it reached character 2316, that this was the first parse error, and the only further input expected from this point onwards is basically termination, it suggests that the JSON messages are being sent fine, and that something else (always, it seems, beginning with a u character) is sent afterwards.

Update: Sometimes have a failure running the Scala 3 integration test, but it happens about 10-20% of the CI runs. CI runs in a clean environment, so when Bloop is presented with a request to compile something for Scala 3, it knows that it has to first compile the “compiler interface” for that version of Scala 3. It usually completes this compilation in the background, caches the result, and then uses the compiled compiler interface to communicate with the Scala 3 compiler and compile whatever we actually asked it to compile.

Unfortunately, there seems to be a race-condition in Bloop which results in some error. Maybe the real compilation is triggered too soon after the compiler interface finishes, and the compiler interface’s class files end up in the wrong place when attempting to start the requested compilation.

In any case, compiling twice works. We actually used this hack in the integration test, but it was removed because this ought to work. So the Scala 3 test should work every time after the first compilation (whether that first compilation succeeds or fails).

In order to diagnose the problem, we would need to set up a test rig which repeatedly runs the Scala 3 test, then deletes the compiler interface and restarts Bloop, before running it again… about 50 times to get a good idea about whether it’s working.

The problem, I believe, lies in Bloop. We might be able to put a workaround into Fury, but it would be great to be able to diagnose the issue as a Bloop problem and have it fixed there. The Bloop configuration file at .bloop/something.json should be enough to test this without involving Fury.

[propensive]

When you say “upgraded to Scala 2.13.2”, what did you actually change? Fury will use its own version of Scala (version 2.12.10, I think), so I think that any change you see is likely to be a consequence of restarting something, or just running the tests a second time…

The issue wasn’t clear from the ticket, (sorry…) so I’ve just updated it.

[anarmanafov1]

Upon taking a closer look I was able to locate the source of the error within the doCCompilation block of Install.scala. When I log the actual exception in doCCompilation(env).recover { case e => I find procname.c:1:10: fatal error: 'sys/prctl.h' file not found.

sys/prctl.h does not exist on mac, only on linux, as explained here: https://github.com/sysown/proxysql/issues/1920

I will need to get linux running to proceed.