Robotframework LS keeps a lot of files open when vendored watchdog is used

Describe the bug Robotframework LS seems to be leaking file handles in some situations in macOS. This has happened to me previously, and now it reappeared on a different macOS install. After it starts leaking file handles this problem seems to persist until a restart. This might be a problem with macOS not freeing the old file handles, but I’m unsure of this.

This happens to me running the LS using CoC-nvim as the LSP client. I can also try to reproduce with other editors if that would be useful for debugging.

To Reproduce Open any robot file, try to trigger any LS feature (for example tab completion.

Expected behavior LS works as usual

Screenshots

lsp: 2020-11-13 13:32:33 UTC pid: 49123 - MainThread - EXCEPTION - robocorp_ls_core.basic
Error calling: ['ps', '-p', '49126', '-o', 'stat'].

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/robotframework_ls/vendored/robocorp_ls_core/basic.py", line 128, in is_process_alive
    process = subprocess.Popen(
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 825, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 1589, in _get_handles
    c2pread, c2pwrite = os.pipe()
OSError: [Errno 23] Too many open files in system

Versions:

• OS: macOS catalina 10.15.7 (19H15)
• Python 3.9
• Robot Framework Version 3.2.2
• Robot Framework Language Server Version 0.5.0
• Client Version :
• coc.nvim version: 0.0.79-cc975fd0ca
• node version: v14.15.0
• vim version: NVIM v0.5.0-nightly-193-g35325ddac

Logs

robotframework_ls.lsp.49123.log

Sometimes when I reproduce this problem, some log files don’t get correctly written (probably related to the file handle problem). I’ll try to get the other log files but it might be a bit difficult.

[fabioz]

Another option would be using polling instead of native file-watching.

i.e.: https://github.com/robocorp/robotframework-lsp/issues/238#issuecomment-789867965

By default the language server will still use watchdog, but now it's also possible to set an environment variable:

ROBOTFRAMEWORK_LS_WATCH_IMPL=fsnotify

to force the usage of polling.

[fabioz]

It seems that the previous version of watchdog (0.10.3) works properly, so, I’m distributing that version with the binary dependencies now to fix this issue.