question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Error from pytest-profiling when doing a 'chdir' in a test

See original GitHub issue

Pytest-profiling (version 1.2.2) is working well on my 2 machines (Windows and Mac OSX).

However, when a test contains a os.chdir, the following error is raised:


self = <pytest_profiling.Profiling object at 0x10c744910>, item = <Function 'test_chdir[dir1]'>

    @pytest.hookimpl(hookwrapper=True)
    def pytest_runtest_call(self, item):
        prof = cProfile.Profile()
        prof.enable()
        yield
        prof.disable()
        prof_filename = os.path.join("prof", clean_filename(item.name) + ".prof")
        try:
>           prof.dump_stats(prof_filename)

/Library/Python/2.7/site-packages/pytest_profiling.py:68: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <cProfile.Profile object at 0x10c780c20>, file = 'prof/test_chdir[dir1].prof'

    def dump_stats(self, file):
        import marshal
>       f = open(file, 'wb')
E       IOError: [Errno 2] No such file or directory: u'prof/test_chdir[dir1].prof'

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/cProfile.py:85: IOError

The code I used to reproduce the issue is the following


import pytest
import os
from time import sleep

@pytest.mark.parametrize("dir", [
	"dir1", "dir2", "NewFolder"
])
def test_chdir(dir):
	try: os.mkdir(dir)
	except: pass
	
	sleep(0.3)

	try:
		os.chdir(dir)
	except Exception as e:
		assert False, "Error while going to directory '{}': {}".format(dir, e)


Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

3reactions
gtalaricocommented, May 13, 2018

Hello @eeaston and thank for maintaining this plugin!

I am having the same issue.

In my case, it is the use of chdir with a temporary directory that causes the error. I have a tool that relies on a getcwd(), so I use a fixture to create a tempdir, chdir into it, and then destroy the tempdir at the end. That causes the .prof to go missing

Here is a min repro:

import pytest
import os
from tempfile import TemporaryDirectory


@pytest.fixture
def tempdir():
    _cwd = os.getcwd()
    with TemporaryDirectory() as path:
        os.chdir(path)
        os.makedirs('folder')
        folderpath = os.path.join(os.getcwd(), 'folder')
        yield folderpath
        # reset cwd before leaving TemDir context since it will will delete `path`
        os.chdir(_cwd)


def test_chdir(tempdir):
    assert os.path.exists(tempdir)

Test w/out --profile passes

(pipenv-pipes-uNZEamAh) ❯❯❯ pytest -k chdir
======================================================================== test session starts =========================================================================
platform darwin -- Python 3.6.5, pytest-3.5.1, py-1.5.3, pluggy-0.6.0 -- /Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/bin/python3.6
cachedir: .pytest_cache
rootdir: /Users/gtalarico/dev/pipenv-pipes, inifile: tox.ini
plugins: profiling-1.3.0, lazy-fixture-0.4.0, cov-2.5.1
collected 49 items / 48 deselected

tests/test_profile.py::test_chdir PASSED

Test w/ profile - Traceback

FileNotFoundError: [Errno 2] No such file or directory: '/private/var/folders/dr/krmtn8_x3x57bm0fs0zbtw800000gn/T/tmpzhvuc0tr/prof/test_chdir.prof'
~/dev/pipenv-pipes master* ⇡
(pipenv-pipes-uNZEamAh) ❯❯❯ pytest -k chdir --profile
======================================================================== test session starts =========================================================================
platform darwin -- Python 3.6.5, pytest-3.5.1, py-1.5.3, pluggy-0.6.0 -- /Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/bin/python3.6
cachedir: .pytest_cache
rootdir: /Users/gtalarico/dev/pipenv-pipes, inifile: tox.ini
plugins: profiling-1.3.0, lazy-fixture-0.4.0, cov-2.5.1
collected 49 items / 48 deselected

tests/test_profile.py::test_chdir PASSED                                                                                                                       [100%]Traceback (most recent call last):
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/bin/pytest", line 11, in <module>
    sys.exit(main())
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/_pytest/config.py", line 61, in main
    return config.hook.pytest_cmdline_main(config=config)
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/pluggy/__init__.py", line 617, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/pluggy/__init__.py", line 222, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/pluggy/__init__.py", line 216, in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/pluggy/callers.py", line 201, in _multicall
    return outcome.get_result()
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
    res = hook_impl.function(*args)
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/_pytest/main.py", line 138, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/_pytest/main.py", line 132, in wrap_session
    exitstatus=session.exitstatus)
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/pluggy/__init__.py", line 617, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/pluggy/__init__.py", line 222, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/pluggy/__init__.py", line 216, in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/pluggy/callers.py", line 196, in _multicall
    gen.send(outcome)
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/_pytest/terminal.py", line 511, in pytest_sessionfinish
    outcome.get_result()
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/pluggy/callers.py", line 76, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/pluggy/callers.py", line 180, in _multicall
    res = hook_impl.function(*args)
  File "/Users/gtalarico/.local/share/virtualenvs/pipenv-pipes-uNZEamAh/lib/python3.6/site-packages/pytest_profiling.py", line 46, in pytest_sessionfinish
    combined = pstats.Stats(self.profs[0])
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pstats.py", line 72, in __init__
    self.init(arg)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pstats.py", line 86, in init
    self.load_stats(arg)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pstats.py", line 99, in load_stats
    with open(arg, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/private/var/folders/dr/krmtn8_x3x57bm0fs0zbtw800000gn/T/tmpzhvuc0tr/prof/test_chdir.prof'
1reaction
eeastoncommented, Feb 20, 2017

@jfthuong can you confirm it works for you now?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Error from pytest-profiling when doing a 'chdir' in a test #40
Pytest-profiling (version 1.2.2) is working well on my 2 machines (Windows and Mac OSX). However, when a test contains a os.chdir, the following...
Read more >
Why does os.chdir generate an error when running doctest in ...
I'm using the testfixtures library for the first time. Using it to test a function that creates a directory. Am getting the below...
Read more >
pytest-profiling - PyPI
Profiling plugin for pytest, with tabular and heat graph output. Tests are profiled with cProfile and analysed with pstats; heat graphs are generated...
Read more >
pytest-profiling - Python Package Health Analysis - Snyk
Tests are profiled with cProfile and analysed with pstats; heat graphs are generated using gprof2dot and dot. Installation. Install using your favourite package ......
Read more >
pytest Documentation - Read the Docs
(See Demo of Python failure reports with pytest). ... pytest rewrites test modules on import by using an import hook to write new...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found