Mainwindow tests leak open file descriptors again

The mainwindow tests leaking open files error from #12328 is back.

Spyder 4.1.4 built fine on openSUSE until some packages got updated a few days ago.

Apply spyder-fdleaks-debug.patch and observe that 4 types of files get opened a lot but not closed properly:

• matplotlibs DejaVuSans.ttf => use matplotlib.testing.decorators.cleanup to make sure all figures are closed?
• ~/.ipython/profile_default/history.sqlite
• /tmp/pytest-spyder-abuild/spyder-py3/pdb_history.sqlite
• /tmp/spyder-abuild/kernel-*.stderr

During the --run-slow run, after a few test_mainwindow.py tests, this happens (7MB logfile!):

[ 1229s] E   OSError: [Errno 24] Too many open files: '/proc/6618/stat'

Same project without the patch: There is no psutil call, so there is a different error. Either the process looks like it is frozen and it is killed after some inactivity or there is a direct abort. The exact failure is not deterministic.

[  510s] spyder/app/tests/test_mainwindow.py::test_pbd_key_leak 
[  510s] (python3:31557): GLib-ERROR **: 23:15:13.726: Creating pipes for GWakeup: Too many open files
[  514s] /var/tmp/rpm-tmp.ALmzUc: line 88: 31557 Trace/breakpoint trap   (core dumped) python3 runtests.py --run-slow -k "not ($skipslowtests_p)" -s --timeout 1800

The current workaround is to completely skip the mainwindow tests, but I’d rather avoid that.

Interestingly but probably unrelated, the most effective fix for a lot of tests after they started to fail a few days ago was to remove the early import of qtpy in runtests.py. A lot of segfaults happen during the tests, if the qtpy import line in runtests.py is not removed, but run fine after removal – despite the strong advise against it in the comment(!)

https://github.com/spyder-ide/spyder/blob/36ff0882cab7677c0ebe650de65b55a8b894a234/runtests.py#L19-L23

Versions

• Spyder version: 4.1.4
• Python version: 3.8.4
• Qt version: 5.15.0
• PyQt version: 5.13.2
• Operating System name/version: openSUSE Tumbleweed on OBS

Dependencies

abuild@greinerT450s:~/rpmbuild/BUILD/spyder-4.1.4> python3
Python 3.8.4 (default, Jul 14 2020, 09:08:56) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from spyder import dependencies
>>> dependencies.declare_dependencies()
>>> print(dependencies.status())

# Mandatory:
atomicwrites >=1.2.0           :  1.4.0 (OK)
chardet >=2.0.0                :  3.0.4 (OK)
cloudpickle >=0.5.0            :  1.4.1 (OK)
diff_match_patch >=20181111    :  20181111 (OK)
intervaltree                   :  None (OK)
IPython >=4.0                  :  7.16.1 (OK)
jedi >=0.17.1                  :  0.17.2 (OK)
keyring                        :  None (OK)
nbconvert >=4.0                :  5.6.1 (OK)
numpydoc >=0.6.0               :  0.9.2 (OK)
parso >=0.7.0                  :  0.7.0 (OK)
pexpect >=4.4.0                :  4.8.0 (OK)
pickleshare >=0.4              :  0.7.5 (OK)
psutil >=5.3                   :  5.7.0 (OK)
pygments >=2.0                 :  2.6.1 (OK)
pylint >=1.0                   :  2.5.3 (OK)
pyls >=0.34.0;<1.0.0           :  0.34.1 (OK)
qdarkstyle >=2.8               :  2.8.1 (OK)
qtawesome >=0.5.7              :  0.7.2 (OK)
qtconsole >=4.6.0              :  4.7.5 (OK)
qtpy >=1.5.0                   :  1.9.0 (OK)
sphinx >=0.6.6                 :  3.1.2 (OK)
spyder_kernels >=1.9.2;<1.10.0 :  1.9.2 (OK)
watchdog                       :  None (OK)
xdg >=0.26                     :  0.26 (OK)
zmq >=17                       :  19.0.1 (OK)

# Optional:
cython >=0.21                  :  0.29.21 (OK)
matplotlib >=2.0.0             :  3.3.0 (OK)
numpy >=1.7                    :  1.19.1 (OK)
pandas >=0.13.1                :  1.0.5 (OK)
scipy >=0.17.0                 :  1.4.1 (OK)
sympy >=0.7.3                  :  1.6 (OK)

More references: #12340 #12534

[bnavigator]

Yes ipykernel is at 5.3.4, I also just created a project with 5.3.2, which is rebuilding currently, but my earlier tests with 5.3.2 did not make a difference: https://build.opensuse.org/project/show/home:bnavigator:spyder-debug2

Paging @impact27 too. He wrote #12534 and presumably knows best about the hacky mainwindow fixture.

[bnavigator]

The easy way to make sure what file is leaking is to:

import psutil proc = psutil.Process() window._initial_number_files = len(proc.open_files()) files = [repr(f) for f in proc.open_files()] and to log files. This should be placed after the yield in the main_window fixture.

Done here:

Code

Patching test_mainwindow.py

import logging
import psutil

code in main_window() after yield:

    p = psutil.Process()
    for f in p.open_files():
        logging.warn(f"file open: {repr(f)}")
    for cp in p.children(recursive=True):
        logging.warn(f"child process: {repr(cp)}")
    for con in p.connections(kind="unix"):
        logging.warn(f"socket: {repr(con)}")

Call

python runtests.py -k "not ($skiptests_p)" --timeout 1800 -s -o log_cli=true  --log-cli-level warn
python runtests.py --run-slow -k "not ($skipslowtests_p)" --timeout 1800 -o log_cli=true  --log-cli-level warn

Buildlog

debug3_log.txt

[  834s] spyder/app/tests/test_mainwindow.py::test_debug_unsaved_file PASSED      [ 10%]
[  834s] ------------------------------ live log teardown -------------------------------
[  834s] WARNING  root:test_mainwindow.py:230 file open: popenfile(path='/tmp/tmpgmus7e9y', fd=17, position=0, mode='r+', flags=688130)
[  834s] 
[  834s] WARNING  root:test_mainwindow.py:230 file open: popenfile(path='/home/abuild/rpmbuild/BUILD/spyder-4.1.4/spyder/images/kite.gif', fd=22, position=114688, mode='r', flags=557056)
[  834s] WARNING  root:test_mainwindow.py:230 file open: popenfile(path='/usr/share/mime/mime.cache', fd=40, position=0, mode='r', flags=557056)
[  834s] WARNING  root:test_mainwindow.py:230 file open: popenfile(path='/home/abuild/rpmbuild/BUILD/spyder-4.1.4/spyder/images/kite.gif', fd=48, position=278528, mode='r', flags=557056)
[  834s] WARNING  root:test_mainwindow.py:230 file open: popenfile(path='/home/abuild/rpmbuild/BUILD/spyder-4.1.4/spyder/images/kite.gif', fd=747, position=0, mode='r', flags=557056)
[  834s] WARNING  root:test_mainwindow.py:230 file open: popenfile(path='/tmp/pytest-spyder-abuild/spyder-py3/pdb_history.sqlite', fd=778, position=0, mode='r+', flags=688130)
[  834s] WARNING  root:test_mainwindow.py:230 file open: popenfile(path='/tmp/pytest-spyder-abuild/spyder-py3/pdb_history.sqlite', fd=835, position=0, mode='r+', flags=688130)
[  834s] WARNING  root:test_mainwindow.py:230 file open: popenfile(path='/tmp/spyder-abuild/kernel-44e2dbad9498.stderr', fd=958, position=0, mode='w', flags=557057)
[  834s] WARNING  root:test_mainwindow.py:232 child process: psutil.Process(pid=6632, name='Xvfb', started='19:39:01')
[  834s] WARNING  root:test_mainwindow.py:232 child process: psutil.Process(pid=11613, name='python3', started='19:46:27')
[  834s] WARNING  root:test_mainwindow.py:232 child process: psutil.Process(pid=11614, name='python3', started='19:46:27')
[  834s] WARNING  root:test_mainwindow.py:232 child process: psutil.Process(pid=11732, name='python3', started='19:46:33')
[  834s] WARNING  root:test_mainwindow.py:232 child process: psutil.Process(pid=11706, name='python3', started='19:46:30')
[  834s] WARNING  root:test_mainwindow.py:234 socket: pconn(fd=462, family=<AddressFamily.AF_UNIX: 1>, type=<SocketKind.SOCK_STREAM: 1>, laddr='', raddr='', status='NONE')

<133 more lines with open sockets>

[  834s] WARNING  root:test_mainwindow.py:234 socket: pconn(fd=758, family=<AddressFamily.AF_UNIX: 1>, type=<SocketKind.SOCK_STREAM: 1>, laddr='', raddr='', status='NONE')
[  834s] WARNING  root:test_mainwindow.py:234 socket: pconn(fd=447, family=<AddressFamily.AF_UNIX: 1>, type=<SocketKind.SOCK_STREAM: 1>, laddr='', raddr='', status='NONE')
[  834s] 
[  836s] spyder/app/tests/test_mainwindow.py::test_runcell[True] ERROR            [ 10%]
[  836s] spyder/app/tests/test_mainwindow.py::test_runcell[True] ERROR            [ 10%]
[  836s] spyder/app/tests/test_mainwindow.py::test_runcell[False] ERROR           [ 11%]
[  837s] spyder/app/tests/test_mainwindow.py::test_runcell[False] ERROR           [ 11%]
[  838s] spyder/app/tests/test_mainwindow.py::test_runcell[False] 
[  838s] (python3:6625): GLib-ERROR **: 19:46:40.938: Creating pipes for GWakeup: Too many open files
[  838s] /var/tmp/rpm-tmp.wUCRTf: line 89:  6625 Trace/breakpoint trap   python3 runtests.py --run-slow -k "not ($skipslowtests_p)" --timeout 1800 -o log_cli=true --log-cli-level warn

Note the growing number of open sockets!