tests/test_user_config.py failure
See original GitHub issuetests/test_user_config.py::test_invalid_config_1
can run into failure when executing the following commands:
pip install pytest-randomly
pytest --randomly-seed=1234
Issue Analytics
- State:
- Created 2 years ago
- Comments:6 (5 by maintainers)
Top Results From Across the Web
No results found
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Steps to reproduce Error
Any of the below commands will cause a test case to fail
pytest tests/test_user_config.py::test_invalid_config_1
pytest tests/test_user_config.py::test_read_config
pytest --randomly-seed=1234 tests/test_user_config.py
(One test case fails randomly)Observations
Test cases in test_user_config.py fail if the order of execution changes. Specifically, if a test case other than
test_default_env_without_config
ortest_empty_config
is executed first, then that test case fails and remaining all pass.Root Cause Analysis
In our
clean
fixture, we are unloadingpdir
related modules in the teardown step. That means, whichever test case executes first,pdir
related import statements are not executed inside that test case because the modules are already available. Since the imports are not executed, the config based on new file path is also not loaded.After the first test case is executed, irrespective of whether it fails or passes, the teardown step in the fixture is executed which unloads the
pdir
modules giving a clean slate for the second test case.test_default_env_without_config
ortest_empty_config
use default config values. So, there’s no affect of config file on them, they always pass.https://github.com/laike9m/pdir2/pull/59 fixes this.
@kshivakumar Sure. If you have any questions, let me know