Large directories cause hang or freeze of Spyder 4.2.1

Issue Report Checklist

• Searched the issues page for similar reports
• Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
• Tried basic troubleshooting (if a bug/error)
  • Restarted Spyder
  • Reset preferences with spyder --reset
  • Reinstalled the latest version of Anaconda
  • Tried the other applicable steps from the Troubleshooting Guide
• Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

On Windows 10, interacting with a cluttered directory can cause hangs that, on my laptop, freeze Spyder for several seconds or permanently.

1. %cd C:\cluttered\directory-magic to switch into that diretory.
2. Run > Run (F5) for any file located in this directory, because this switches the working directory to C:\cluttered\directory.
3. Once the working directory of the console is the large directory:
   1. Saving a file, that is located in the large directory.
   2. Autosaving of a file that is located in the large directory.
   3. Executing any shell commands with iPython’s ! prefix.
   4. Any interaction with the file pane. In this case the delay may be so large, that Spyder is for all intents and purposes crashed.

I first observed the issue after saving a scratch script to %TEMP% which at the time container more than 11,000 entries, but the issue is reproducible with any similarly cluttered directory. Such directories can occur e.g. with simulation software.

Windows Defender (and other resident antivirus packages) can cause file-operations to be slowed down enough to make File Explorer react slowly in large directories. In this case, disabling Windows Defender didn’t help.

On Linux the issue was not reproducible.

I suspect that the issue is caused by refreshing the Files pane, even if it isn’t visible.

What steps reproduce the problem?

Direct demonstration. Save the following script in an empty directory, and execute it with Run > Run (F5). It should cause Spyder to hang for a long time, or crash outright. Alternatively it might just cause the same second-scale hang as other described operations.

for i in range(10000):
    with open(f"a.{i:4d}", "w") as fout:
        print(i, file=fout)

Interactive use.

1. Create directory C:\tmp and fill it with 10,000 files.
2. In Spyder, save a file as C:\tmp\a.py.
3. Run > Run (F5). Spyder should now hang for a moment.
4. File > Save (Ctrl+S). Spyder should now hang for a moment.
5. Seemingly randomly while editing there will be further hangs, whenever Spyder autosaves.
6. Open the file pane. Might become effectively unrecoverable.

What is the expected output? What do you see instead?

Not applicable.

Paste Traceback/Error Below (if applicable)

Not applicable.

Versions

• Spyder 4.2.1
• Python 3.8.5 64-bit
• Qt 5.12.10
• PyQt5 5.12.3
• Windows 10 Version 2004 (OS Build 19041.804)

Dependencies

# Mandatory:
atomicwrites >=1.2.0            :  1.4.0 (OK)
chardet >=2.0.0                 :  4.0.0 (OK)
cloudpickle >=0.5.0             :  1.6.0 (OK)
diff_match_patch >=20181111     :  20200713 (OK)
intervaltree >=3.0.2            :  3.1.0 (OK)
IPython >=7.6.0                 :  7.20.0 (OK)
jedi =0.17.2                    :  0.17.2 (OK)
jsonschema >=3.2.0              :  3.2.0 (OK)
keyring >=17.0.0                :  22.0.1 (OK)
nbconvert >=4.0                 :  6.0.7 (OK)
numpydoc >=0.6.0                :  1.1.0 (OK)
paramiko >=2.4.0                :  2.7.2 (OK)
parso =0.7.0                    :  0.7.0 (OK)
pexpect >=4.4.0                 :  4.8.0 (OK)
pickleshare >=0.4               :  0.7.5 (OK)
psutil >=5.3                    :  5.8.0 (OK)
pygments >=2.0                  :  2.7.4 (OK)
pylint >=1.0                    :  2.6.0 (OK)
pyls >=0.36.2;<1.0.0            :  0.36.2 (OK)
pyls_black >=0.4.6              :  0.4.6 (OK)
pyls_spyder >=0.3.0             :  0.3.0 (OK)
qdarkstyle >=2.8                :  2.8.1 (OK)
qtawesome >=0.5.7               :  1.0.1 (OK)
qtconsole >=5.0.1               :  5.0.2 (OK)
qtpy >=1.5.0                    :  1.9.0 (OK)
rtree >=0.8.3                   :  0.9.4 (OK)
setuptools >=39.0.0             :  52.0.0.post20210125 (OK)
sphinx >=0.6.6                  :  3.4.3 (OK)
spyder_kernels >=1.10.1;<1.11.0 :  1.10.1 (OK)
textdistance >=4.2.0            :  4.2.1 (OK)
three_merge >=0.1.1             :  0.1.1 (OK)
watchdog >=0.10.3               :  1.0.2 (OK)
zmq >=17                        :  20.0.0 (OK)

# Optional:
cython >=0.21                   :  0.29.21 (OK)
matplotlib >=2.0.0              :  3.3.2 (OK)
numpy >=1.7                     :  1.19.2 (OK)
pandas >=1.1.1                  :  1.2.1 (OK)
scipy >=0.17.0                  :  1.6.0 (OK)
sympy >=0.7.3                   :  1.7.1 (OK)
\0

[eafpres]

@eafpres could you deactivate the Files plugin and check again? (for that go to Preferences > Plugins > Internal plugins and uncheck Files) The thing is that even if the pane is not visible the plugin is still loaded and could be trying to create/load the model that represents the directory you are chaging your working directory on the console.

Let us know if disabling the Files plugin helps!

I confirm this resolves my issue.

[ccordoba12]

Agreed, that should solve your problem @eafpres.