Refactor pytest discovery code in extension

Environment data

• VS Code version: 1.30.1 (dea8705087adb1b5e5ae1d9123278e178656186a)
• Extension version (available under the Extensions sidebar): 2018.12.1
• OS and version: macOS 10.14.2 (18C54)
• Python version (& distribution if applicable, e.g. Anaconda): 3.6.8 (Anaconda)
• Type of virtual environment used : conda
• Relevant/affected Python packages and their versions: pytest 4.1.0

Local setting is:

{
    "python.pythonPath": "/Users/dror/anaconda3/envs/tmp/bin/python",
    "python.unitTest.pyTestArgs": [
        "."
    ],
    "python.unitTest.unittestEnabled": false,
    "python.unitTest.nosetestsEnabled": false,
    "python.unitTest.pyTestEnabled": true
}

Expected behaviour

When running Python: Discover Unit Tests I expect not to receive an error message.

Actual behaviour

When I run pytest at the root of my project (from the terminal) everything works as expected: all tests are found and executed. However, when I try to run Python: Discover Unit Tests, I get the following error No tests discovered, please check the configuration settings for the tests.. The output is odd:

============================= test session starts ==============================
platform darwin -- Python 3.6.8, pytest-4.1.0, py-1.7.0, pluggy-0.8.0
rootdir: /Users/dror/tmp, inifile:
collected 2 items
<Module test_foo.py>
  <Function test_a>
  <Function test_b>

========================= no tests ran in 0.01 seconds =========================```

Moreover, when I do Python: Run All Tests I get the above output, followed by actually running the tests and passing them (as if I was running pytest directly in the console).

Steps to reproduce:

1. Create a conda virtual environment (tmp) and install pytest using pip
2. Create foo.py and test_foo.py (see below).
3. Run pytest in the console and see the two tests pass.
4. In vscode, run Python: Discover Unit Tests.
5. Then run Python: Run All Unit Tests.

#  foo.py
def bar(x):
    return x + 10

############################################

#  test_foo.py 
import foo


def test_a():
    assert 1 == 1


def test_b():
    assert foo.bar(5) == 15

Logs

Output for Python in the Output: see above.

The output of running the tests is:

============================= test session starts ==============================
platform darwin -- Python 3.6.8, pytest-4.1.0, py-1.7.0, pluggy-0.8.0
rootdir: /Users/dror/tmp, inifile:
collected 2 items
<Module test_foo.py>
  <Function test_a>
  <Function test_b>

========================= no tests ran in 0.01 seconds =========================
============================= test session starts ==============================
platform darwin -- Python 3.6.8, pytest-4.1.0, py-1.7.0, pluggy-0.8.0
rootdir: /Users/dror/tmp, inifile:
collected 2 items

test_foo.py ..                                                           [100%]

 generated xml file: /var/folders/w6/6m9g0bjn1q17bqdw33mqdqb40000gp/T/tmp-20946jC8DYWY6C8bR.xml 
=========================== 2 passed in 0.01 seconds ===========================

Output from Console under the Developer Tools panel: I couldn’t find relevant information here.

[brettcannon]

@drorata we plan to solve it in the extension. Basically pytest does not guarantee stdout format stability so we have to change on our end.

[kondratyev-nv]

I have fixed a similar issue in test explorer extension for Python. I have used the approach similar to the one used for unittest discovery https://github.com/DonJayamanne/pythonVSCode/blob/master/src/client/unittests/unittest/services/parserService.ts. To discover tests I used pytest_collection_finish hook, see https://github.com/kondratyev-nv/vscode-python-test-adapter/pull/48/files#diff-8fe02f614ffe36db100b8ad8758546b3R24.

Hope this might be of help!