Incremental compiler crashes because of open file handles

I searched StackOverflow and GitHub and found no issues like this, so here we go:

steps

I am running SBT 1.0.1.

I am developing a fairly small Akka HTTP service (which should have nothing to do with the issue but for context). My workflow is to have an SBT shell open and ~compile continuously running.

Today this happened:

[error] Total time: 1 s, completed 5-Sep-2017 3:23:30 PM
[error] java.io.IOException: User limit of inotify instances reached or too many open files
[error]         at sun.nio.fs.LinuxWatchService.<init>(LinuxWatchService.java:64)
[error]         at sun.nio.fs.LinuxFileSystem.newWatchService(LinuxFileSystem.java:47)
[error]         at sbt.Defaults$.$anonfun$globalSbtCore$20(Defaults.scala:250)
[error]         at sbt.Defaults$$anon$3.watchService(Defaults.scala:561)
[error]         at sbt.Watched$.executeContinuously(Watched.scala:92)
[error]         at sbt.BasicCommands$.$anonfun$continuous$2(BasicCommands.scala:215)
[error]         at sbt.CommandUtil$.withAttribute(CommandUtil.scala:42)
[error]         at sbt.BasicCommands$.$anonfun$continuous$1(BasicCommands.scala:213)
[error]         at sbt.Command$.$anonfun$applyEffect$4(Command.scala:125)
[error]         at sbt.Command$.$anonfun$applyEffect$2(Command.scala:121)
[error]         at sbt.MainLoop$.processCommand(MainLoop.scala:121)
[error]         at sbt.MainLoop$.$anonfun$next$2(MainLoop.scala:105)
[error]         at sbt.State$$anon$1.runCmd$1(State.scala:228)
[error]         at sbt.State$$anon$1.process(State.scala:234)
[error]         at sbt.MainLoop$.$anonfun$next$1(MainLoop.scala:105)
[error]         at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error]         at sbt.MainLoop$.next(MainLoop.scala:105)
[error]         at sbt.MainLoop$.run(MainLoop.scala:98)
[error]         at sbt.MainLoop$.$anonfun$runWithNewLog$1(MainLoop.scala:76)
[error]         at sbt.io.Using.apply(Using.scala:22)
[error]         at sbt.MainLoop$.runWithNewLog(MainLoop.scala:70)
[error]         at sbt.MainLoop$.runAndClearLast(MainLoop.scala:52)
[error]         at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:37)
[error]         at sbt.MainLoop$.runLogged(MainLoop.scala:29)
[error]         at sbt.StandardMain$.runManaged(Main.scala:104)
[error]         at sbt.xMain.run(Main.scala:71)
[error]         at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
[error]         at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
[error]         at xsbt.boot.Launch$.run(Launch.scala:109)
[error]         at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
[error]         at xsbt.boot.Launch$.launch(Launch.scala:117)
[error]         at xsbt.boot.Launch$.apply(Launch.scala:18)
[error]         at xsbt.boot.Boot$.runImpl(Boot.scala:41)
[error]         at xsbt.boot.Boot$.main(Boot.scala:17)
[error]         at xsbt.boot.Boot.main(Boot.scala)
[error] java.io.IOException: User limit of inotify instances reached or too many open files
[error] Use 'last' for the full log.

Now, unfortunately I derped up and just restarted my sbt and ~compile without pulling the full log…

My max allowed number of file handles is 512k.

λ sysctl fs.inotify.max_user_watches
fs.inotify.max_user_watches = 524288

The number of files in my repo is far smaller than that (44 of these files are non-target/)

λ find . -type f | wc -l
168

problem

Clearly, SBT should not die like this 😃

expectation

That ~compile just keeps going

notes

sbt version: 1.0.1

Point me in a direction and I will try to provide more information. And if this happens again, I will pull the full log as well.

[Duhemm]

Thanks for the report @NeQuissimus. That’s surprising, I’ve tested it on fairly large codebases, akka/akka, scala/scala and apache/spark, without hitting that issue.

In the meantime, I thing you can workaround the issue with:

> set watchService := (() => new sbt.io.PollingWatchService(pollInterval.value))

[eed3si9n]

Closing this for now since sbt 1.3.x has reimplemented watch.