Language server fails to load pylsp_black plugin in macOS app

Problem Description

pylsp fails to load the pylsp_black plugin in the macOS standalone application. This does not appear to afflict Spyder installed from pip or conda, only the standalone application. I have not verified the Windows application. There are two issues:

• pathspec is not included in the bundle
• black cannot be imported

In the lsp server log:

WARNING - pylsp.config.config - Failed to load pylsp entry point 'pylsp_black': The 'pathspec>=0.9.0' distribution was not found and is required by black

This indicates that the pathspec package is not included in the application bundle and is easily remedied by adding it to INCLUDES. However, upon doing so, a subsequent issue arises:

WARNING - pylsp.config.config - Failed to load pylsp entry point 'pylsp_black': No module named '610faff656c4cfcbb4a3__mypyc'

From the Spyder internal console, one can verify that the pylsp_black plugin is unable to load the black module.

>>> import pkg_resources
>>> [ep for ep in pkg_resources.iter_entry_points('pylsp') if ep.name=='pylsp_black'][0].load()
Traceback (most recent call last):
  File "code.pyc", line 90, in runcode
  File "<console>", line 2, in <module>
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/pkg_resources/__init__.py", line 2458, in load
    return self.resolve()
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/pkg_resources/__init__.py", line 2464, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/Users/rclary/Documents/Python/spyder/installers/macOS/dist/Spyder.app/Contents/Resources/lib/python3.9/pylsp_black/plugin.py", line 7, in <module>
    import black
ModuleNotFoundError: No module named '610faff656c4cfcbb4a3__mypyc'
>>> import black
Traceback (most recent call last):
  File "code.pyc", line 90, in runcode
  File "<console>", line 2, in <module>
ModuleNotFoundError: No module named '610faff656c4cfcbb4a3__mypyc'

Further investigation is required to remedy this last issue.

Versions

• Spyder version: 5.3.0 None
• Python version: 3.9.5 64-bit
• Qt version: 5.15.2
• PyQt5 version: 5.15.6
• Operating System: Darwin 20.6.0

Dependencies

# Mandatory:
applaunchservices >=0.1.7            :  0.2.1 (OK)
atomicwrites >=1.2.0                 :  1.4.0 (OK)
chardet >=2.0.0                      :  4.0.0 (OK)
cloudpickle >=0.5.0                  :  2.0.0 (OK)
cookiecutter >=1.6.0                 :  1.7.3 (OK)
diff_match_patch >=20181111          :  20200713 (OK)
intervaltree >=3.0.2                 :  3.1.0 (OK)
IPython >=7.31.1;<8.0.0              :  7.32.0 (OK)
jedi >=0.17.2;<0.19.0                :  0.18.1 (OK)
jellyfish >=0.7                      :  0.9.0 (OK)
jsonschema >=3.2.0                   :  4.4.0 (OK)
keyring >=17.0.0                     :  23.5.0 (OK)
nbconvert >=4.0                      :  6.4.5 (OK)
numpydoc >=0.6.0                     :  1.2.1 (OK)
parso >=0.7.0;<0.9.0                 :  0.8.3 (OK)
pexpect >=4.4.0                      :  4.8.0 (OK)
pickleshare >=0.4                    :  0.7.5 (OK)
psutil >=5.3                         :  5.9.0 (OK)
pygments >=2.0                       :  2.11.2 (OK)
pylint >=2.5.0                       :  2.13.3 (OK)
pyls_spyder >=0.4.0                  :  0.4.0 (OK)
pylsp >=1.4.1;<1.5.0                 :  1.4.1 (OK)
pylsp_black >=1.2.0                  :  1.2.0 (OK)
qdarkstyle >=3.0.2;<3.1.0            :  3.0.3 (OK)
qstylizer >=0.1.10                   :  0.2.1 (OK)
qtawesome >=1.0.2                    :  1.1.1 (OK)
qtconsole >=5.3.0;<5.4.0             :  5.3.0 (OK)
qtpy >=2.0.1                         :  2.0.1 (OK)
rtree >=0.9.7                        :  0.9.7 (OK)
setuptools >=49.6.0                  :  60.10.0 (OK)
sphinx >=0.6.6                       :  4.5.0 (OK)
spyder_kernels >=2.3.0;<2.4.0        :  2.3.0 (OK)
textdistance >=4.2.0                 :  4.2.2 (OK)
three_merge >=0.1.1                  :  0.1.1 (OK)
watchdog >=0.10.3                    :  2.1.7 (OK)
zmq >=17                             :  22.3.0 (OK)

# Optional:
cython >=0.21                        :  0.29.28 (OK)
matplotlib >=3.0.0                   :  3.5.1 (OK)
numpy >=1.7                          :  1.22.3 (OK)
pandas >=1.1.1                       :  1.4.1 (OK)
scipy >=0.17.0                       :  1.8.0 (OK)
sympy >=0.7.3                        :  1.10.1 (OK)

# Spyder plugins:
spyder_terminal.terminalplugin 1.2.2 :  1.2.2 (OK)

[mrclary]

So including blib2to3 in PACKAGES, and 610faff656c4cfcbb4a3__mypyc and pathspec in INCLUDES, resolves the issues. I’ve submitted a new recipe for black to py2app ronaldoussoren/py2app#418. It should be resolved by #17612. And, yes, I’ll inquire on the psf/black repo.

[ccordoba12]

You could ask on the Black repo about it or add the file that ends in __mypyc instead.