Tests not Reporting Status if One of the Test Folder are Outside of Workspace Folder

Environment data

• VS Code version: 1.35 (Insiders)
• Extension version (available under the Extensions sidebar): 2019.4.12954
• OS and version: Ubuntu 18.04 LTS (interrupter is running inside of Debian 9 container via Remote Container feature)
• Python version (& distribution if applicable, e.g. Anaconda): 3.7
• Type of virtual environment used (N/A | venv | virtualenv | conda | …): Remote Container, system Python
• Relevant/affected Python packages and their versions: pytest?

Expected behavior

After running tests via PyTest, the status symbol next to them should change from a ? to either based on test success or failure.

Actual behavior

After running tests via PyTest, the status symbol never stop showing as ?.

Steps to reproduce:

1. Create Python project with tests
2. Create a second Python project with tests next to the first one
3. Open the second Python project as project folder in VS Code
4. Configure PyTest to run tests for both projects

The use case for this is that we have shared “framework” of code that all of our projects are built that has a set of test that get run to make sure our code does not break the framework. I do not want VS Code running full linting/code checks on the framework because that is not the code I am working on. Just need to make sure tests pass.

Ex. (assuming 1. is at /project1 and 2. is at /project2, and /project2 is the folder you have open in VS Code)

{
    "python.testing.pyTestEnabled": true,
    "python.testing.pyTestArgs": [
       "/project1",
       "/project2",
    ],
}

Also, if project1 is not pip install -e in your environment, you will need to set a $PYTHONPATH environment variable.

Logs

Nothing useful. Just a lot of “Reloading modules… done. Analysis restarted.” Python Test Log has nothing but standard PyTest output, no types of errors. Console under Developer tools has nothing useful enough, just “[Extension Host] Python Extension: Cached data exists getEnvironmentVariables, <workspace_path>”

[AngellusMortis]

I figured out this is two pronged. If you have pytest.ini in a parent directory (outside of workspace) or you have tests that runs outside of workspace, it causes test results to not show up correctly.

I will try to slap together a sample repo this weekend if I can.

[karrtikr]

The use case for this is that we have shared “framework” of code that all of our projects are built that has a set of test that get run to make sure our code does not break the framework. I do not want VS Code running full linting/code checks on the framework because that is not the code I am working on. Just need to make sure tests pass. https://github.com/AngellusMortis/pytest-vscode

In this case your project is actually pytest-vscode which contains the code you want to test, so I think the correct thing should be opening pytest-vscode as a workspace instead. We no longer run any linting checks/code checks unless you open the file so it shouldn’t be a problem. Let me know if you disagree and we’ll re-evaluate opening the issue.

cc @karthiknadig