Changes to .env not respected when running tests

Environment data

• VS Code version: 1.35.0

• Extension version (available under the Extensions sidebar): XXX

• OS and version: Pop!OS (Ubuntu 18.10) / Linux x64 4.18.0-21-generic

• Python version: Python 2.7.15+ from distro

• Type of virtual environment used: virtualenv

• Relevant/affected Python packages and their versions:

  • pytest==4.6.1

• Language Server: Jedi

Expected behaviour

Debugging a test, either from the test explorer or from the test code lens decoration, should respect the python.testing.pyTestPath setting

Actual behaviour

When debugging tests, VS.Code appears to behave like python.testing.pyTestPath is not set. It seems to run the pytest module directly.

Steps to reproduce:

• Create directory (say vscode_pytest)
• Fire vs.code inside it
• Create a file called test_me.py with the following contents:

# test_me.py
import os

def test_me():
    assert os.environ.get('TEST_ENV') == '1'

• Create a virtualenv inside the workspace called venv with virtualenv venv

• When VS.Code detects this new virtualenv at ./venv, accept it as the project default.

• Invoke Python: Configure Tests and select pytest.

• Check that it discovered the test_me.py file and the test_me test inside.

• Run the tests and check that they fail

• Create a file called pytest.sh with the following contents:

#!/bin/bash
PYTEST=$(dirname $0)/venv/bin/pytest
TEST_ENV=1 exec $PYTEST "$@"

• Make it executable and check that running makes the tests pass

• Edit workspace settings and set:

  • "python.testing.pyTestPath": "${workspaceFolder}/pytest.sh"

• Run the tests in the Test Explorer tool

  • They should pass

• Debug the test using the debug icon in Test Explorer

  • The 'DEBUG CONSOLE` output should show that they fail.

Considerations

In a way, this issue relates to #4647:

The VS.Code Python test setup allows selecting the path to the pytest ("python.testing.pyTestPath") script and the arguments passed to it ("python.testing.pyTestArgs").

But it does not allow specifying environment variables, so I set the pytest script just so the right environment variables are set when the test is run.

If there was a "python.testing.pyTestEnv": {} setting, it would have solved my needs without the need to specify a pyTestPath.

On the other hand, those three settings look redundant with the settings for tasks in launch.json.

In my opinion, instead of specifying pyTestPathor pyTestArgs, the testing infrastructure should allow specifying one of the "type": "python" tasks in launch.json.

The Python: Configure Tests command could even pre-fill such a task inside launch.json if one doesn’t exist (or migrate one from the settings of pyTestPath and pyTestArgs on first run, which could then be deprecated). The the only setting needed would be:

• ``“python.testing.pyTestTask”: “”`

[leorochael]

Indeed, reloading the window fixed the problem with loading the environment.

But I need to reload the window whenever .env changes when running tests with the test sidebar (but not when debugging the test, apparently), so it seems you’re right there is a bug.

[drcrook1]

I believe I have this same problem. When debugging tests it can find my environment variables set in a dev.env; however, when running the tests (not debugging); the variables are not available.

================================== FAILURES =================================== ________________ TestClassifiedWidgetsProvider.test_get_widget ________________

self = <test_classified_widgets.TestClassifiedWidgetsProvider object at 0x0000021AA02A22B0>

def test_get_widget(self):
    """
    Test to ensure we can get a widget
    """

  cursor = data_helpers.get_db_cxn()

tests\test_providers\test_classified_widgets.py:19:

---

tests\helpers\data_helpers.py:10: in get_db_cxn driver = os.environ[“SQL_DRIVER”]

---

self = environ({‘ALLUSERSPROFILE’: ‘C:\ProgramData’, ‘AMD_ENTRYPOINT’: ‘vs/workbench/services/extensions/node/extensionHostP…RRENT_TEST’: ‘tests/test_providers/test_classified_widgets.py::TestClassifiedWidgetsProvider::test_get_widget (call)’}) key = ‘SQL_DRIVER’

when debugging this works fine.