Failed to abort an infinite loop

(This might be related to my unusual settings described in #102)

mutmut got stuck on a mutation, and didn’t time it out. Reproduction:

$ mktmpenv -p python3.7
Running virtualenv with interpreter /usr/local/bin/python3.7
Using base prefix '/usr/local/pythonz/pythons/CPython-3.7.2'
/usr/local/pythonz/pythons/CPython-2.7.14/lib/python2.7/site-packages/virtualenv.py:1047: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
New python executable in /usr/local/virtualenvs/tmp-b4da94a5634daf06/bin/python3.7
Also creating executable in /usr/local/virtualenvs/tmp-b4da94a5634daf06/bin/python
Installing setuptools, pip, wheel...done.
Requirement already up-to-date: pip in /usr/local/virtualenvs/tmp-b4da94a5634daf06/lib/python3.7/site-packages (19.0.3)
Requirement already up-to-date: setuptools in /usr/local/virtualenvs/tmp-b4da94a5634daf06/lib/python3.7/site-packages (40.8.0)
This is a temporary environment. It will be deleted when you run 'deactivate'.

$ git clone [e-mail]:nedbat/cog.git
Cloning into 'cog'...
remote: Enumerating objects: 657, done.
remote: Counting objects: 100% (657/657), done.
remote: Compressing objects: 100% (230/230), done.
remote: Total 657 (delta 458), reused 619 (delta 421), pack-reused 0
Receiving objects: 100% (657/657), 172.29 KiB | 574.00 KiB/s, done.
Resolving deltas: 100% (458/458), done.

$ cd cog

$ git checkout c1f605d7aaf3750568b7e7a5d784c081fff0fc97
Note: checking out 'c1f605d7aaf3750568b7e7a5d784c081fff0fc97'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at c1f605d mutmut settings

$ pip install tox==3.7.0
Collecting tox==3.7.0
  Using cached https://files.pythonhosted.org/packages/4a/bd/50cd91221508db5bb66e211ac327d0da4a5db6d0db5ad35d6fbf1e4af5dc/tox-3.7.0-py2.py3-none-any.whl
Collecting six<2,>=1.0.0 (from tox==3.7.0)
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=30.0.0 in /usr/local/virtualenvs/tmp-b4da94a5634daf06/lib/python3.7/site-packages (from tox==3.7.0) (40.8.0)
Collecting pluggy<1,>=0.3.0 (from tox==3.7.0)
  Using cached https://files.pythonhosted.org/packages/84/e8/4ddac125b5a0e84ea6ffc93cfccf1e7ee1924e88f53c64e98227f0af2a5f/pluggy-0.9.0-py2.py3-none-any.whl
Collecting filelock<4,>=3.0.0 (from tox==3.7.0)
  Downloading https://files.pythonhosted.org/packages/d7/ca/3c74396a9ed8a4cfab5459800edeef9a1269591cb21f5a49bd71a49c5fa2/filelock-3.0.10-py3-none-any.whl
Collecting virtualenv>=1.11.2 (from tox==3.7.0)
  Downloading https://files.pythonhosted.org/packages/33/5d/314c760d4204f64e4a968275182b7751bd5c3249094757b39ba987dcfb5a/virtualenv-16.4.3-py2.py3-none-any.whl (2.0MB)
    100% |████████████████████████████████| 2.0MB 62kB/s
Collecting toml>=0.9.4 (from tox==3.7.0)
  Using cached https://files.pythonhosted.org/packages/a2/12/ced7105d2de62fa7c8fb5fce92cc4ce66b57c95fb875e9318dba7f8c5db0/toml-0.10.0-py2.py3-none-any.whl
Collecting py<2,>=1.4.17 (from tox==3.7.0)
  Using cached https://files.pythonhosted.org/packages/76/bc/394ad449851729244a97857ee14d7cba61ddb268dce3db538ba2f2ba1f0f/py-1.8.0-py2.py3-none-any.whl
Installing collected packages: six, pluggy, filelock, virtualenv, toml, py, tox
Successfully installed filelock-3.0.10 pluggy-0.9.0 py-1.8.0 six-1.12.0 toml-0.10.0 tox-3.7.0 virtualenv-16.4.3

$ tox -e py27,py36
py27 create: /usr/local/virtualenvs/tmp-b4da94a5634daf06/cog/.tox/py27
py27 installdeps: coverage, pytest
py27 develop-inst: /usr/local/virtualenvs/tmp-b4da94a5634daf06/cog
py27 installed: DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.,atomicwrites==1.3.0,attrs==19.1.0,-e [e-mail]:nedbat/cog.git@c1f605d7aaf3750568b7e7a5d784c081fff0fc97#egg=cogapp,coverage==4.5.3,funcsigs==1.0.2,more-itertools==5.0.0,pathlib2==2.3.3,pluggy==0.9.0,py==1.8.0,pytest==4.3.1,scandir==1.10.0,six==1.12.0
py27 run-test-pre: PYTHONHASHSEED='110312279'
py27 runtests: commands[0] | coverage run -m pytest
..............................................................................................................................                           [100%]
126 passed in 1.07 seconds
py36 create: /usr/local/virtualenvs/tmp-b4da94a5634daf06/cog/.tox/py36
py36 installdeps: coverage, pytest
py36 develop-inst: /usr/local/virtualenvs/tmp-b4da94a5634daf06/cog
py36 installed: atomicwrites==1.3.0,attrs==19.1.0,-e [e-mail]:nedbat/cog.git@c1f605d7aaf3750568b7e7a5d784c081fff0fc97#egg=cogapp,coverage==4.5.3,more-itertools==6.0.0,pluggy==0.9.0,py==1.8.0,pytest==4.3.1,six==1.12.0
py36 run-test-pre: PYTHONHASHSEED='110312279'
py36 runtests: commands[0] | coverage run -m pytest
..............................................................................................................................                           [100%]
126 passed in 1.01 seconds
___________________________________________________________________________ summary ____________________________________________________________________________
  py27: commands succeeded
  py36: commands succeeded
  congratulations :)

$ pip install mutmut
Collecting mutmut
  Downloading https://files.pythonhosted.org/packages/cc/fe/2e96bc00df659baa47d46f3574ce38d90e3af1e52bded16bdfdcbf4d8fbb/mutmut-1.3.1.tar.gz
Collecting glob2 (from mutmut)
  Downloading https://files.pythonhosted.org/packages/f0/e8/970c7a031b2d7f9a21fefaa8c9d5c38001f8f25055f4ffcb32b3dbecd1ea/glob2-0.6.tar.gz
Collecting parso (from mutmut)
  Downloading https://files.pythonhosted.org/packages/19/b1/522b2671cc6d134c9d3f5dfc0d02fee07cab848e908d03d2bffea78cca8f/parso-0.3.4-py2.py3-none-any.whl (93kB)
    100% |████████████████████████████████| 102kB 240kB/s
Collecting tri.declarative (from mutmut)
  Downloading https://files.pythonhosted.org/packages/14/93/64cabe646f27563fa48877a1d78f7bc0b31a460ffc8eba0be5cc46ca59f7/tri.declarative-1.2.1.tar.gz
Collecting click (from mutmut)
  Downloading https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl (81kB)
    100% |████████████████████████████████| 81kB 172kB/s
Collecting pony (from mutmut)
  Downloading https://files.pythonhosted.org/packages/fa/28/7106ba263cc206bee67afdb3f92975c024875b3b8f194b5a3038293763c8/pony-0.7.9.tar.gz (267kB)
    100% |████████████████████████████████| 276kB 206kB/s
Collecting junit-xml==1.8 (from mutmut)
  Downloading https://files.pythonhosted.org/packages/a6/2a/f8d5aab80bb31fcc789d0f2b34b49f08bd6121cd8798d2e37f416df2b9f8/junit-xml-1.8.tar.gz
Collecting tri.struct (from tri.declarative->mutmut)
  Downloading https://files.pythonhosted.org/packages/a6/d3/e2d04e73251850bf7f4d552e38356c20a8555034434a1a7717088f0c2255/tri.struct-2.5.7.tar.gz
Requirement already satisfied: six in /usr/local/virtualenvs/tmp-b4da94a5634daf06/lib/python3.7/site-packages (from junit-xml==1.8->mutmut) (1.12.0)
Building wheels for collected packages: mutmut, glob2, tri.declarative, pony, junit-xml, tri.struct
  Building wheel for mutmut (setup.py) ... done
  Stored in directory: /Users/ned/Library/Caches/pip/wheels/62/08/4e/9df18ef82cb934549753981bdb1388adba6bf389aac751abbf
  Building wheel for glob2 (setup.py) ... done
  Stored in directory: /Users/ned/Library/Caches/pip/wheels/1b/31/14/5a25a14e933674d8636bcc42af811c7eb87ea8f6e947c17273
  Building wheel for tri.declarative (setup.py) ... done
  Stored in directory: /Users/ned/Library/Caches/pip/wheels/d0/46/58/d03eec161d4de3eb451ce9ec673a831aafbdfaf6196669bed1
  Building wheel for pony (setup.py) ... done
  Stored in directory: /Users/ned/Library/Caches/pip/wheels/0e/b6/52/61130f0fffc57a62b94b36bf4807ec77b436e41c7bd7e17887
  Building wheel for junit-xml (setup.py) ... done
  Stored in directory: /Users/ned/Library/Caches/pip/wheels/c6/c5/65/5b166afb6eac87dc300368ec4d92f39502dd41cdc73056d49e
  Building wheel for tri.struct (setup.py) ... done
  Stored in directory: /Users/ned/Library/Caches/pip/wheels/c8/90/ae/a32ab1ed04cf48c87331f44710649a7c187ba2bf8eefaad891
Successfully built mutmut glob2 tri.declarative pony junit-xml tri.struct
Installing collected packages: glob2, parso, tri.struct, tri.declarative, click, pony, junit-xml, mutmut
Successfully installed click-7.0 glob2-0.6 junit-xml-1.8 mutmut-1.3.1 parso-0.3.4 pony-0.7.9 tri.declarative-1.2.1 tri.struct-2.5.7

$ mutmut run

- Mutation testing starting -

These are the steps:
1. A full test suite run will be made to make sure we
   can run the tests successfully and we know how long
   it takes (to detect infinite loops for example)
2. Mutants will be generated and checked

Mutants are written to the cache in the .mutmut-cache
directory. Print found mutants with `mutmut results`.

Legend for output:
🎉 Killed mutants. The goal is for everything to end up in this bucket.
⏰ Timeout. Test suite took 10 times as long as the baseline so were killed.
🤔 Suspicious. Tests took a long time, but not long enough to be fatal.
🙁 Survived. This means your tests needs to be expanded.

mutmut cache is out of date, clearing it...
1. Running tests without mutations
⠴ Running... Done

2. Checking mutants
⠙ 309/464  🎉 267  ⏰ 0  🤔 0  🙁 42^C
Aborted!

$ mutmut show 309
--- cogapp/cogapp.py
+++ cogapp/cogapp.py
@@ -512,7 +512,7 @@
                 # them, compute the md5 hash of the old output.
                 previous = ""
                 hasher = hashlib.md5()
-                while l and not self.isEndOutputLine(l):
+                while l and  self.isEndOutputLine(l):
                     if self.isBeginSpecLine(l):
                         raise CogError("Unexpected '%s'" % self.options.sBeginSpec,
                             file=sFileIn, line=fIn.linenumber())

$ more setup.cfg
[tool:pytest]
addopts = -q -rfe

[mutmut]
paths_to_mutate = cogapp/cogapp.py
runner = sh -c '.tox/py27/bin/pytest -x && .tox/py36/bin/pytest -x'

It took about 5 minutes for mutmut to run the first 308 mutants, and I waited about 5 minutes before Ctrl-C’ing number 309.

[ghost]

Just to add my two cents: I also observed that the readline call on the stdout file handle blocks the entire main mutmut process. From my cursory understanding of the code, the stdout is only consumed to enable the progress spinner.

I simply patched my copy to remove the “readline” call completely and I am now just calling the callback every 0.05s while the subprocess is running:

    while process.returncode is None:
        try:
            if os.name == 'nt':  # pragma: no cover
                line = stdout.readline()
                # windows gives readline() raw stdout as a b''
                # need to decode it
                line = line.decode("utf-8")
                if line:  # ignore empty strings and None
                    callback(line.rstrip())
            else:
                #line = stdout.readline()
                #if not line:
                #    break
                callback('.')
                sleep(0.05)

You can also now adjust the spinner’s smoothness and are no longer dependent on your test runner to give you a smooth 20Hz spinner.

[atugushev]

Here is the fix in #171. Any reviews and tests are welcomed! Thanks! 🙏