"mutmut run 65" doesn't seem to re-test a mutation?
See original GitHub issueMaybe this is just me not understanding what “mutmut run 65” is supposed to do. I updated a test to kill mutant 65, then ran “mutmut run 65”, but it showed it as still surviving. When I then applied 65, the tests failed:
$ mktmpenv -p python3.7
Running virtualenv with interpreter /usr/local/bin/python3.7
Using base prefix '/usr/local/pythonz/pythons/CPython-3.7.2'
/usr/local/pythonz/pythons/CPython-2.7.14/lib/python2.7/site-packages/virtualenv.py:1047: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
New python executable in /usr/local/virtualenvs/tmp-b4da94a5634daf06/bin/python3.7
Also creating executable in /usr/local/virtualenvs/tmp-b4da94a5634daf06/bin/python
Installing setuptools, pip, wheel...done.
Requirement already up-to-date: pip in /usr/local/virtualenvs/tmp-b4da94a5634daf06/lib/python3.7/site-packages (19.0.3)
Requirement already up-to-date: setuptools in /usr/local/virtualenvs/tmp-b4da94a5634daf06/lib/python3.7/site-packages (40.8.0)
This is a temporary environment. It will be deleted when you run 'deactivate'.
$ git clone git@github.com:nedbat/cog.git
Cloning into 'cog'...
remote: Enumerating objects: 657, done.
remote: Counting objects: 100% (657/657), done.
remote: Compressing objects: 100% (230/230), done.
remote: Total 657 (delta 458), reused 619 (delta 421), pack-reused 0
Receiving objects: 100% (657/657), 172.29 KiB | 574.00 KiB/s, done.
Resolving deltas: 100% (458/458), done.
$ cd cog
$ git checkout c1f605d
HEAD is now at c1f605d mutmut settings
$ mutmut run
- Mutation testing starting -
These are the steps:
1. A full test suite run will be made to make sure we
can run the tests successfully and we know how long
it takes (to detect infinite loops for example)
2. Mutants will be generated and checked
Mutants are written to the cache in the .mutmut-cache
directory. Print found mutants with `mutmut results`.
Legend for output:
🎉 Killed mutants. The goal is for everything to end up in this bucket.
⏰ Timeout. Test suite took 10 times as long as the baseline so were killed.
🤔 Suspicious. Tests took a long time, but not long enough to be fatal.
🙁 Survived. This means your tests needs to be expanded.
mutmut cache is out of date, clearing it...
1. Running tests without mutations
⠴ Running... Done
2. Checking mutants
⠋ 309/464 🎉 267 ⏰ 0 🤔 0 🙁 42^C
Aborted! # this is because of the stuck infinite loop described in #103
$ mutmut show 65
--- cogapp/cogapp.py
+++ cogapp/cogapp.py
@@ -187,7 +187,7 @@
def msg(self, s):
self.prout("Message: "+s)
- def out(self, sOut='', dedent=False, trimblanklines=False):
+ def out(self, sOut='', dedent=True, trimblanklines=False):
""" The cog.out function.
"""
if trimblanklines and ('\n' in sOut):
$ mutmut run 65
- Mutation testing starting -
These are the steps:
1. A full test suite run will be made to make sure we
can run the tests successfully and we know how long
it takes (to detect infinite loops for example)
2. Mutants will be generated and checked
Mutants are written to the cache in the .mutmut-cache
directory. Print found mutants with `mutmut results`.
Legend for output:
🎉 Killed mutants. The goal is for everything to end up in this bucket.
⏰ Timeout. Test suite took 10 times as long as the baseline so were killed.
🤔 Suspicious. Tests took a long time, but not long enough to be fatal.
🙁 Survived. This means your tests needs to be expanded.
1. Using cached time for baseline tests, to run baseline again delete the cache file
2. Checking mutants
⠹ 1/1 🎉 0 ⏰ 0 🤔 0 🙁 1
# The mutant should still be surviving here, I haven't done anything yet.
# Now apply the change to the test.
$ git checkout 667fe10
Previous HEAD position was c1f605d mutmut settings
HEAD is now at 667fe10 Include dedent=default in a test
$ mutmut run 65
- Mutation testing starting -
These are the steps:
1. A full test suite run will be made to make sure we
can run the tests successfully and we know how long
it takes (to detect infinite loops for example)
2. Mutants will be generated and checked
Mutants are written to the cache in the .mutmut-cache
directory. Print found mutants with `mutmut results`.
Legend for output:
🎉 Killed mutants. The goal is for everything to end up in this bucket.
⏰ Timeout. Test suite took 10 times as long as the baseline so were killed.
🤔 Suspicious. Tests took a long time, but not long enough to be fatal.
🙁 Survived. This means your tests needs to be expanded.
1. Using cached time for baseline tests, to run baseline again delete the cache file
2. Checking mutants
⠹ 1/1 🎉 0 ⏰ 0 🤔 0 🙁 1
# Shouldn't this have been a killed mutant now?
$ mutmut apply 65
$ tox -e py36
py36 develop-inst-nodeps: /usr/local/virtualenvs/tmp-b4da94a5634daf06/cog
py36 installed: atomicwrites==1.3.0,attrs==19.1.0,-e git+git@github.com:nedbat/cog.git@667fe101e97eaec6bad022fd4fb162d96ade392a#egg=cogapp,coverage==4.5.3,more-itertools==6.0.0,pluggy==0.9.0,py==1.8.0,pytest==4.3.1,six==1.12.0
py36 run-test-pre: PYTHONHASHSEED='2230429556'
py36 runtests: commands[0] | coverage run -m pytest
...F.......................................................................................................................... [100%]
=========================================================================== FAILURES ===========================================================================
______________________________________________________________ CogTestsInMemory.testCogOutDedent _______________________________________________________________
self = <cogapp.test_cogapp.CogTestsInMemory testMethod=testCogOutDedent>
def testCogOutDedent(self):
infile = """\
//[[[cog
cog.out("This is the first line\\n")
cog.out('''
This is dedent=True
''', dedent=True, trimblanklines=True)
cog.out('''
This is dedent=False
''', dedent=False, trimblanklines=True)
cog.out('''
This is dedent=default
''', trimblanklines=True)
cog.out("This is the last line\\n")
//]]]
This is the first line
This is dedent=True
This is dedent=False
This is dedent=default
This is the last line
//[[[end]]]
"""
infile = reindentBlock(infile)
> self.assertEqual(Cog().processString(infile), infile)
E AssertionError: '//[[[398 chars]lse\nThis is dedent=default\nThis is the last [15 chars]]]\n' != '//[[[398 chars]lse\n This is dedent=default\nThis is the l[19 chars]]]\n'
E //[[[cog
E cog.out("This is the first line\n")
E cog.out('''
E This is dedent=True
E ''', dedent=True, trimblanklines=True)
E cog.out('''
E This is dedent=False
E ''', dedent=False, trimblanklines=True)
E cog.out('''
E This is dedent=default
E ''', trimblanklines=True)
E cog.out("This is the last line\n")
E //]]]
E This is the first line
E This is dedent=True
E This is dedent=False
E - This is dedent=default
E + This is dedent=default
E ? ++++
E This is the last line
E //[[[end]]]
cogapp/test_cogapp.py:170: AssertionError
=================================================================== short test summary info ====================================================================
FAILED cogapp/test_cogapp.py::CogTestsInMemory::testCogOutDedent
1 failed, 125 passed in 1.13 seconds
ERROR: InvocationError for command '/usr/local/virtualenvs/tmp-b4da94a5634daf06/cog/.tox/py36/bin/coverage run -m pytest' (exited with code 1)
___________________________________________________________________________ summary ____________________________________________________________________________
ERROR: py36: commands failed
$ git diff
diff --git a/cogapp/cogapp.py b/cogapp/cogapp.py
index cb8486d..51122e0 100644
--- a/cogapp/cogapp.py
+++ b/cogapp/cogapp.py
@@ -187,7 +187,7 @@ def evaluate(self, cog, globals, fname):
def msg(self, s):
self.prout("Message: "+s)
- def out(self, sOut='', dedent=False, trimblanklines=False):
+ def out(self, sOut='', dedent=True, trimblanklines=False):
""" The cog.out function.
"""
if trimblanklines and ('\n' in sOut):
$
Issue Analytics
- State:
- Created 5 years ago
- Comments:8 (6 by maintainers)
Top Results From Across the Web
mutmut - python mutation tester — mutmut documentation
You can stop the mutation run at any time and mutmut will restart where you left off. It's also smart enough to retest...
Read more >Mutmut - Ned Batchelder
The theory is that a mutation will change the behavior of your program, so if your test suite is testing closely enough, some...
Read more >Myocardial Structural Alteration and Systolic Dysfunction in ...
To evaluate the presence of myocardial structural alterations and subtle myocardial dysfunction during familial screening in asymptomatic ...
Read more >ACC/AHA Guidelines for the Clinical Application of ...
The guidelines include recommendations for the use of Doppler echocardiography in both specific cardiovascular disorders and in the evaluation of patients with ...
Read more >Clinical Case Studies for the Family Nurse Practitioner
The rash does not appear to cause discomfort for Sarah. ... Family m edical h istory: PGF (age 65): Type 2 diabetes mellitus;...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I would think that “mutmut run <id>” should always run the tests. If I just wanted to see the results, why wouldn’t I use “mutmut show <id>”?
Well… if mutmut thinks the tests and code is unchanged, maybe you as a user are confused, or have mutmut configured wrong. In those cases I don’t think it would help the situation to just blindly rerun the test.