Pytest - Run test from test explorer fail with uppercase test folder name

Environment data

• VS Code version: 1.35.1
• Extension version (available under the Extensions sidebar): 2019.6.22090
• OS and version: Windows 10 Home Edition
• Python version (& distribution if applicable, e.g. Anaconda): Python 3.6.8
• Type of virtual environment used (N/A | venv | virtualenv | conda | …): global python 3.6.8
• Relevant/affected Python packages and their versions: pytest 4.6.3
• Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info #3977): Jedi

Expected behaviour

Test file could be run from test explorer in activity bar.

Actual behaviour

Test file could not run from test explorer in activity bar.

CodeLens adornments disappear from the test file.

Steps to reproduce:

1. enable pytest from D:\projectRoot.vscode\setting.json as following:

{
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "Test"
    ]
}

1. project folder structure is as following:

D:\projectRoot\
    .vscode\
    SRC\
        src.py
    Test\
        __init__.py
        test_src.py

1. Run Test file from test explorer in activity bar

Logs

Python Test Log say that:

============================= test session starts =============================
platform win32 -- Python 3.6.8, pytest-4.6.3, py-1.5.2, pluggy-0.12.0
rootdir: d:\projectRoot
collected 0 items / 1 errors

=================================== ERRORS ====================================
______________________ ERROR collecting test/test_src.py ______________________
ImportError while importing test module 'd:\projectRoot\test\test_src.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
ModuleNotFoundError: No module named 'test.test_src'
...

Additional Information

All the setting work for the vscode-python for previous release (2019.5.18875).

When I change the folder name containing the test file from Test to test, with python.testing.pytestArgs setting in setting.json to [“test”], test file could be run from test explorer. However, codeLens adornments still disappear.

It seems for the current release, vscode-python change the uppercase letter for folder name to lowercase before they import the test module. This change to letter case cause the problem.

[ericsnowcurrently]

This should be resolved now due to #6781 and #6877. Those changes are already in the insiders build of the extension and will be part of the September release.

[PhilMacKay]

After thorough investigation, I realized this happens with pytest version 5.0.0 and upper, and the extension’s TestCollector (in pythonFiles/testing_tools/adapter/pytest/_discovery.py) which applies os.path.normcase on every path. According to pytest’s change log https://docs.pytest.org/en/latest/changelog.html#id15, bugfix https://github.com/pytest-dev/pytest/issues/2761#issue-256094577 changes the way they deal with paths on case-insensitive systems.

Basically, the workarounds I found was:

• Use most recent version of pytest and VSCode Python extension version 2019.5.18875 --> everything works as it should. OR
• Use most recent version of VSCode Python extension and a pytest version below 5.0.0. In that case, we suffer from bug #6695 and capital letters in file paths are lost, but we can run any test individually. OR
• Use most recent version of both VSCode Python extension and pytest, but modify the file ~/.vscode/extensions/ms-python.python-2019.X.XXXXX/pythonFiles/testing_tools/adapter/pytest/_discovery.py line 64: add an exception on windows such that NORMCASE maps to os.path.normpath instead of os.path.normcase. It solves all my problems, but I suspect it could create a bunch of problems in other situations 😏.