question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

When a unittest subTest fails, it says that no tests ran

See original GitHub issue

Environment data

  • VS Code version: 1.33.1
  • Extension version (available under the Extensions sidebar): 2019.4.11987
  • OS and version: Ubuntu 16.04
  • Python version (& distribution if applicable, e.g. Anaconda): 3.5.2
  • Type of virtual environment used (N/A | venv | virtualenv | conda | …): virtualenv
  • Relevant/affected Python packages and their versions: N/A

Expected behaviour

Expected to see the tests fail, and have VS Code show that it failed.

Actual behaviour

Does not indicate if the test succeeded or failed, even though the logs clearly indicate that the test was run and it failed. The status bar at the bottom of the screen says “No Tests Run”. A warning notification pops up saying that no tests were run.

Steps to reproduce:

  1. Create an empty folder and open it in VS Code
  2. Create a file named test_failing_subtests.py with the contents:
import unittest
class TestFailingSubtests(unittest.TestCase):
	def test_failing_subtest(self):
		with self.subTest():
			self.fail()
	def test_failing_subtest_with_msg(self):
		with self.subTest("Placeholder message"):
			self.fail()
if __name__ == '__main__':
	unittest.main()
  1. Set up the VS Code configuration so that it discovers the test
  2. In the command palette, run Python: Discover Tests
  3. In the command palette, run Python: Run All Tests

Logs

Output for Python Test Log in the Output panel (ViewOutput, change the drop-down the upper-right of the Output panel to Python Test Log)

start
test_fail_subtests.TestFailingSubtests.test_failing_subtest
test_fail_subtests.TestFailingSubtests.test_failing_subtest_with_msg
test_failing_subtest (test_fail_subtests.TestFailingSubtests) ... test_failing_subtest_with_msg (test_fail_subtests.TestFailingSubtests) ... 
======================================================================
FAIL: test_failing_subtest (test_fail_subtests.TestFailingSubtests) (<subtest>)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/vscode-python-subtest-bug/test_fail_subtests.py", line 7, in test_failing_subtest
    self.fail()
AssertionError: None

======================================================================
FAIL: test_failing_subtest_with_msg (test_fail_subtests.TestFailingSubtests) [Placeholder message]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/vscode-python-subtest-bug/test_fail_subtests.py", line 11, in test_failing_subtest_with_msg
    self.fail()
AssertionError: None

----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=2)

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)

[Extension Host] Python Extension: Cached data exists ActivatedEnvironmentVariables, /tmp/vscode-python-subtest-bug
console.ts:134 [Extension Host] Python Extension: getActivatedEnvironmentVariables, Class name = S, Arg 1: <Uri:/tmp/vscode-python-subtest-bug>, Arg 2: undefined, Arg 3: undefined
console.ts:134 [Extension Host] Python Extension: Cached data exists getEnvironmentVariables, /tmp/vscode-python-subtest-bug
console.ts:134 [Extension Host] Python Extension: Cached data exists ActivatedEnvironmentVariables, /tmp/vscode-python-subtest-bug
console.ts:134 [Extension Host] Python Extension: getActivatedEnvironmentVariables, Class name = S, Arg 1: <Uri:/tmp/vscode-python-subtest-bug>, Arg 2: undefined, Arg 3: undefined
console.ts:134 [Extension Host] Python Extension: Cached data exists getEnvironmentVariables, /tmp/vscode-python-subtest-bug

I’d be happy to take a stab at fixing this if somebody could point me in the right direction.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:8 (3 by maintainers)

github_iconTop GitHub Comments

3reactions
ItachiSancommented, May 20, 2020

Any updates on this?

0reactions
kimadelinecommented, Oct 21, 2021

Closing as duplicate of #17561.

Read more comments on GitHub >

github_iconTop Results From Across the Web

unittest subTest failure causes result to be omitted from listing
msg256580 ‑ (view) Author: Zachary Ware (zach.ware) * Date: 2015‑12‑17 05:49 msg256588 ‑ (view) Author: Martin Panter (martin.panter) * Date: 2015‑12‑17 07:45 msg256590 ‑ (view)...
Read more >
Continuing in Python's unittest when an assertion fails
An failing assert that does not fail the test is a bad idea. It makes your tests overly complicated (which lowers readability and ......
Read more >
Subtests in Python - Paul Ganssle
Without the self.subTest context manager, this test would fail immediately when i=1 and execution would end, reporting that test_loop has ...
Read more >
Python unittest subtest
By using the subTest() context manager, the test didn't stop after the first failure. Also, it shows a very detailed message after each...
Read more >
Test::More - yet another framework for writing test scripts
Should an ok() fail, it will produce some diagnostics: not ok 18 - sufficient mucus # Failed ... No tests will be run,...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found