Nose Tests discovery fails when tests are split by folders and imported to __init__.py (wantModule and wantDirectory are not parsed)

Environment data

• VS Code version: 1.37.1
• Extension version (available under the Extensions sidebar): 2019.8.30787
• OS and version: Windows 10 running VS Code, remote plugin connects to Ubuntu 18.04.2 via SSH
• Python version (& distribution if applicable, e.g. Anaconda): python 2.7.15rc1
• Type of virtual environment used (N/A | venv | virtualenv | conda | …): virtualenv
• Relevant/affected Python packages and their versions: Django==1.11.23; django-nose==1.4.6; nose==1.3.7
• Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info #3977): jedi

Expected behaviour

Test discovery for nose should scan for both wantFile and wantModule and/or wantDirectory log outputs

Actual behaviour

We have a django project with django-nose test runner. Tests are split into modules, and all tests are imported to __init__.py. I think these are some obsolete tests, from default django test runner. But nose runs them perfectly.

I create a simple script, that translates the nosetests call to ./manage.py test call. Basically it just strips -vvv and replaces it with --verbosity

#!/bin/bash
function arg_remove ()
{
    local array=("${@:3}")

    for ((i=0; i<"${#array[@]}"; ++i)); do
        case ${array[i]} in
            "$2") unset array[i]; break ;;
        esac
    done

    # clean up unset array indexes
    for i in "${!array[@]}"; do
        new_array+=( "${array[i]}" )
    done
    array=("${new_array[@]}")
    unset new_array

    # assign array outside function scope
    local -g "$1=(  )"
        eval ${1}='("${array[@]}")'
}
arg_remove ARG_PASS "-vvv" "$@"
echo $@
direnv exec /home/igor/myproject/backend/ /home/igor/.virtualenvs/myproject/bin/python /home/igor/myproject/backend/manage.py test --settings=core.local_settings --verbosity=3 -l nose.selector --nologcapture "${ARG_PASS[@]}"

The structure of tests is (stripped)

File backend/apps/tests/__init__.py

from .action import ActionTestCase

File backend/apps/tests/action.py

from apps.myproj.tests.base import BaseTestCase


class ActionMSFTestCase(BaseTestCase):
    pass
    # stripped

The output of --collect-only command is different to what vscode-python expects.

Relevant output to tests above is:

nose.selector: DEBUG: wantDirectory /home/igor/ata_portal/backend/apps/api/tests? True
nose.selector: DEBUG: Test name /home/igor/ata_portal/backend/apps/api/tests resolved to file /home/igor/ata_portal/backend/apps/api/tests, module None, call None
nose.selector: DEBUG: Final resolution of test name /home/igor/ata_portal/backend/apps/api/tests: file /home/igor/ata_portal/backend/apps/api/tests module apps.api.tests call None
nose.selector: DEBUG: wantModule <module 'apps.api.tests' from '/home/igor/ata_portal/backend/apps/api/tests/__init__.pyc'>? True
nose.selector: DEBUG: wantClass <class 'apps.api.tests.action_msf.ActionMSFTestCase'>? True

parserService here scans only for wantFile output.

In case above it founds wantClass above any wantFile ... py? True line. That causes testFile.suites.push(testSuite); to fail

Steps to reproduce:

1. Create a package tests
2. Add a module action.py to package with some real test case
3. import action inside tests/__init__.py
4. Configure nose tests for workspace
5. Run Python: Discover Tests command

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

Test Discovery failed: 
TypeError: Cannot read property 'suites' of undefined

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)

notificationsAlerts.ts:40 Test discovery error, please check the configuration settings for the tests.
onDidNotificationChange @ notificationsAlerts.ts:40
console.ts:137 [Extension Host] Python Extension: displayDiscoverStatus TypeError: Cannot read property 'suites' of undefined
	at t.forEach.t (/home/igor/.vscode-server/extensions/ms-python.python-2019.8.30787/out/client/extension.js:75:1015174)
	at Array.forEach (<anonymous>)
	at h.parseNoseTestModuleCollectionResult (/home/igor/.vscode-server/extensions/ms-python.python-2019.8.30787/out/client/extension.js:75:1014343)
	at e.split.forEach (/home/igor/.vscode-server/extensions/ms-python.python-2019.8.30787/out/client/extension.js:75:1013968)
	at Array.forEach (<anonymous>)
	at h.getTestFiles (/home/igor/.vscode-server/extensions/ms-python.python-2019.8.30787/out/client/extension.js:75:1013849)
	at h.parse (/home/igor/.vscode-server/extensions/ms-python.python-2019.8.30787/out/client/extension.js:75:1013661)
	at h.discoverTests (/home/igor/.vscode-server/extensions/ms-python.python-2019.8.30787/out/client/extension.js:75:1012881)
	at process._tickCallback (internal/process/next_tick.js:68:7)

[IBestuzhev]

Thanks @ericsnowcurrently

Yes, I understand it’s quite rare issue, the project is legacy and so it uses not recent versions.

Was this working on an earlier version of nose?

Never checked before.

what version of nose are you using?

nose==1.3.7

what happens when you run the nose directly in the terminal? what is the command (with args) that you use to do so?

That is Django project, so I can’t run nosetests command directly. It fails to import most of the tests and they require django.setup() to be called.

The error is AppRegistryNotReady: Apps aren't loaded yet. And I think it’s not relevant to this issue.

But I can run python manage.py test, and this command does some django setup and then runs nosetests --verbosity=2 --with-id

So for test discovery I run ./manage.py test --settings=core.local_settings --verbosity=3 -l nose.selector --nologcapture --collect-only

how is the bash script you included above related to running the tests?

It translates nosetests call to python manage.py test command

what is in your workspace settings.json?

"python.testing.cwd": "/home/igor/project/backend",
"python.testing.nosetestPath": "/home/igor/custom_nose",  # custom bash script
"python.testing.promptToConfigure": false,

[karthiknadig]

Closing via #16371