question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Error when running debugging using pytest in IDE with geopandas

See original GitHub issue

I ran into an error today while running a test in debug mode using PyCharm (I found the same issue on VS Code as well) when geopandas is imported. Outside of debug mode it seems to work fine. I’ve also found that downgrading pytest to 6.2.4 resolves the issue.

I was able to reproduce the problem by setting up a test environment using conda: A simple test script with a breakpoint set at the assert statement produces the error below

conda create --name test --channel=conda-forge python pytest geopandas

Today when I set it up that command resulted in the following version being installed (full list of packages installed below): python=3.10.2 pytest=7.1.1 geopandas=0.10.2

I am using an ARM Mac running macOS 12.3, I’ve also gotten the same result on an Intel Mac macOS 11.6.4.

import geopandas


def test_pytest_and_geopandas():
    assert False

A simple test script with a breakpoint set at the assert statement like that above produces the following error if the test is run in debug mode in PyCharm:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/_pytest/main.py", line 268, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/_pytest/main.py", line 321, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/_pytest/main.py", line 332, in pytest_collection
INTERNALERROR>     session.perform_collect()
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/_pytest/main.py", line 644, in perform_collect
INTERNALERROR>     self.ihook.pytest_collectreport(report=rep)
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/teamcity/pytest_plugin.py", line 328, in pytest_collectreport
INTERNALERROR>     self.report_test_failure(test_id, report)
INTERNALERROR>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/teamcity/pytest_plugin.py", line 223, in report_test_failure
INTERNALERROR>     self.ensure_test_start_reported(test_id)
INTERNALERROR>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/teamcity/pytest_plugin.py", line 186, in ensure_test_start_reported
INTERNALERROR>     self.teamcity.testStarted(test_id, flowId=test_id, captureStandardOutput=capture_standard_output, metainfo=metainfo)
INTERNALERROR>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_runner_tools.py", line 181, in testStarted
INTERNALERROR>     commands = _TREE_MANAGER_HOLDER.manager.level_opened(self._test_to_list(testName), _write_start_message)
INTERNALERROR>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_runner_tools.py", line 51, in manager
INTERNALERROR>     self._fill_manager()
INTERNALERROR>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_runner_tools.py", line 59, in _fill_manager
INTERNALERROR>     from _jb_serial_tree_manager import SerialTreeManager
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 945, in _find_spec
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 92, in find_spec
INTERNALERROR>     if self._early_rewrite_bailout(name, state):
INTERNALERROR>   File "/Users/aengel/miniconda3/envs/test/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 193, in _early_rewrite_bailout
INTERNALERROR>     path = PurePath(os.path.sep.join(parts) + ".py")
INTERNALERROR> AttributeError: module 'posixpath' has no attribute 'sep'

or VS Code

tests[/test_addl.py:10](): in <module>
    import geopandas
.[./../miniconda3/envs/test/lib/python3.10/site-packages/geopandas/__init__.py:1](): in <module>
    from geopandas._config import options  # noqa
.[./../miniconda3/envs/test/lib/python3.10/site-packages/geopandas/_config.py:109](): in <module>
    default_value=_default_use_pygeos(),
.[./../miniconda3/envs/test/lib/python3.10/site-packages/geopandas/_config.py:95](): in _default_use_pygeos
    import geopandas._compat as compat
.[./../miniconda3/envs/test/lib/python3.10/site-packages/geopandas/_compat.py:9](): in <module>
    import pyproj
.[./../miniconda3/envs/test/lib/python3.10/site-packages/pyproj/__init__.py:93](): in <module>
    pyproj.network.set_ca_bundle_path()
.[./../miniconda3/envs/test/lib/python3.10/site-packages/pyproj/network.py:51](): in set_ca_bundle_path
    ca_bundle_path = certifi.where()
.[./../miniconda3/envs/test/lib/python3.10/site-packages/certifi/core.py:36](): in where
    _CACERT_CTX = get_path("certifi", "cacert.pem")
.[./../miniconda3/envs/test/lib/python3.10/importlib/resources.py:119](): in path
    reader = _common.get_resource_reader(_common.get_package(package))
.[./../miniconda3/envs/test/lib/python3.10/importlib/_common.py:52](): in get_resource_reader
    return reader(spec.name)  # type: ignore
<frozen importlib._bootstrap_external>:548: in _check_name_wrapper
    ???
<frozen importlib._bootstrap_external>:1081: in get_resource_reader
    ???
<frozen importlib._bootstrap>:1027: in _find_and_load
    ???
<frozen importlib._bootstrap>:1002: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:945: in _find_spec
    ???
.[./../miniconda3/envs/test/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:92](): in find_spec
    if self._early_rewrite_bailout(name, state):
.[./../miniconda3/envs/test/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:193](): in _early_rewrite_bailout
    path = PurePath(os.path.sep.join(parts) + ".py")
E   AttributeError: module 'posixpath' has no attribute 'sep'
Package            Version
------------------ ------------
attrs              21.4.0
branca             0.4.2
brotlipy           0.7.0
certifi            2021.10.8
cffi               1.15.0
charset-normalizer 2.0.12
click              8.0.4
click-plugins      1.1.1
cligj              0.7.2
cryptography       36.0.1
cycler             0.11.0
Fiona              1.8.21
folium             0.12.1.post1
fonttools          4.31.1
GDAL               3.4.2
geopandas          0.10.2
idna               3.3
iniconfig          1.1.1
Jinja2             3.0.3
joblib             1.1.0
kiwisolver         1.4.0
mapclassify        2.4.3
MarkupSafe         2.1.1
matplotlib         3.5.1
munch              2.5.0
munkres            1.1.4
networkx           2.7.1
numpy              1.22.3
packaging          21.3
pandas             1.4.1
Pillow             9.0.1
pip                22.0.4
pluggy             1.0.0
py                 1.11.0
pycparser          2.21
pyOpenSSL          22.0.0
pyparsing          3.0.7
pyproj             3.3.0
PySocks            1.7.1
pytest             7.1.1
python-dateutil    2.8.2
pytz               2021.3
requests           2.27.1
Rtree              0.9.7
scikit-learn       1.0.2
scipy              1.8.0
setuptools         60.10.0
Shapely            1.8.0
six                1.16.0
threadpoolctl      3.1.0
tomli              2.0.1
unicodedata2       14.0.0
urllib3            1.26.9
wheel              0.37.1
xyzservices        2022.3.0

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:6
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
K0Tecommented, Mar 29, 2022

Hi,

I have found the same bug after updating project from Python3.9 to Python3.10. Looks like proj==3.3.0 works with Python3.9 debugger and with Python3.10 without debugger, but breaks with 3.10 and debugger. This does not seem to be related to pytest, some code completely breaks os.path, pytest is just the first one to find that attribute is missing. Minimal reproducible example:

from os import path

print(path.sep) # prints '/'
# First comment out _pyproj_global_context_initialize() call in
# pyproj/__init__.py, otherwise it will fail in import
from pyproj import _pyproj_global_context_initialize

print(path.sep) # prints '/'

_pyproj_global_context_initialize() # break os.path

print(path.sep) # fails

Stack-trace:

=========================================================================================== test session starts ============================================================================================
collected 0 items / 1 error                                                                                                                                                                                
##teamcity[testCount timestamp='2022-03-29T12:59:39.969' count='0']
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/_pytest/main.py", line 322, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/_pytest/main.py", line 333, in pytest_collection
INTERNALERROR>     session.perform_collect()
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/_pytest/main.py", line 634, in perform_collect
INTERNALERROR>     self.items.extend(self.genitems(node))
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/_pytest/main.py", line 812, in genitems
INTERNALERROR>     node.ihook.pytest_collectreport(report=rep)
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/teamcity/pytest_plugin.py", line 328, in pytest_collectreport
INTERNALERROR>     self.report_test_failure(test_id, report)
INTERNALERROR>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/teamcity/pytest_plugin.py", line 223, in report_test_failure
INTERNALERROR>     self.ensure_test_start_reported(test_id)
INTERNALERROR>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/teamcity/pytest_plugin.py", line 186, in ensure_test_start_reported
INTERNALERROR>     self.teamcity.testStarted(test_id, flowId=test_id, captureStandardOutput=capture_standard_output, metainfo=metainfo)
INTERNALERROR>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_runner_tools.py", line 181, in testStarted
INTERNALERROR>     commands = _TREE_MANAGER_HOLDER.manager.level_opened(self._test_to_list(testName), _write_start_message)
INTERNALERROR>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_runner_tools.py", line 51, in manager
INTERNALERROR>     self._fill_manager()
INTERNALERROR>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pycharm/_jb_runner_tools.py", line 59, in _fill_manager
INTERNALERROR>     from _jb_serial_tree_manager import SerialTreeManager
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 945, in _find_spec
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 91, in find_spec
INTERNALERROR>     if self._early_rewrite_bailout(name, state):
INTERNALERROR>   File "/Users/kote/portside/phenom/.env/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 195, in _early_rewrite_bailout
INTERNALERROR>     path = PurePath(os.path.sep.join(parts) + ".py")
INTERNALERROR> AttributeError: module 'posixpath' has no attribute 'sep'
1reaction
Zac-HDcommented, Mar 20, 2022

Looks like a bug to me: we should also let PurePath do the joining, not hand-rolling string manipulation.

(and the error is because there’s no such thing as os.path.sep, it’s just os.sep… or os.pathsep, but confusingly that’s an entirely different thing.)

Read more comments on GitHub >

github_iconTop Results From Across the Web

Running pytest with "debug in test" works correctly but "run in ...
I have a project where I am running tests using the debug menu pull down and they run fine. ... When I do...
Read more >
IntelliJ Python plugin pytest code runs but debug doesn't work
I have an issue debugging python tests using pytest on IntelliJ IDE with the python plugin. the tests can be run (they have...
Read more >
Debugging — Kedro 0.18.4 documentation
This guide provides examples on how to instantiate a post-mortem debugging session with pdb using Kedro Hooks when an uncaught error occurs during...
Read more >
Running Rust on Microcontrollers - mbedded.ninja
Architecture Support; MCU Family Support; IDEs, Programming and the Debugging Experience; RTOSes; Rust Disadvantages. Lets jump straight in!
Read more >
conda install geopandas failed with initial frozen solve
I resolved the problem by installing geopandas using ... Currently, I am running conda install geopandas and it is not working, but I...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found