UpdateOperatorMutator is causing Karma to disconnect

Hello,

I am using Stryker on an AngularJS project with the following npm modules:

• “grunt-stryker”: “0.5.2”,
• “stryker”: “0.6.2”,
• “stryker-api”: “0.5.3”,
• “stryker-html-reporter”: “0.4.2”,
• “stryker-jasmine”: “0.2.2”,
• “stryker-karma-runner”: “0.4.1”

Whenever Stryker mutates native Javscript for/while loops that use increment/decrement operators, those mutations cause Karma to disconnect and mark all remaining mutations as ‘survived’. The error message i see in the console looks like:

Mutation testing 31% (ETC 480s) [27 killed] [0 survived] [0 no coverage] [0 timeout] [0 error] [2017-08-11 15:09:58.742] [TRACE] IsolatedTestRunnerAdapter - 11 08 2017 15:09:58.741:INFO [framework.browserify]: bundle built

[2017-08-11 15:10:01.059] [TRACE] IsolatedTestRunnerAdapter - 11 08 2017 15:10:01.060:WARN [Chrome 56.0.2924 (Windows 10 0.0.0)]: Disconnected (1 times)

[2017-08-11 15:10:01.060] [TRACE] IsolatedTestRunnerAdapter - 11 08 2017 15:10:01.061:INFO [karma]: Restarting Chrome 56.0.2924 (Windows 10 0.0.0) (1 of 5 attempts)

[2017-08-11 15:10:01.068] [TRACE] IsolatedTestRunnerAdapter - [2017-08-11 15:10:01.068] [INFO] KarmaTestRunner - karma run done with 1

[2017-08-11 15:10:01.070] [DEBUG] EventRecorderReporter - Writing event onMutantTested to file reports\mutation\events\00030-onMutantTested.json [2017-08-11 15:10:01.078] [TRACE] IsolatedTestRunnerAdapter - No captured browser, open http:

[2017-08-11 15:10:01.079] [TRACE] IsolatedTestRunnerAdapter - [2017-08-11 15:10:01.079] [INFO] KarmaTestRunner - karma run done with 1

[2017-08-11 15:10:01.080] [DEBUG] EventRecorderReporter - Writing event onMutantTested to file reports\mutation\events\00031-onMutantTested.json [2017-08-11 15:10:01.087] [TRACE] IsolatedTestRunnerAdapter - No captured browser, open http://

[2017-08-11 15:10:01.088] [TRACE] IsolatedTestRunnerAdapter - [2017-08-11 15:10:01.089] [INFO] KarmaTestRunner - karma run done with 1

[2017-08-11 15:10:01.090] [DEBUG] EventRecorderReporter - Writing event onMutantTested to file reports\mutation\events\00032-onMutantTested.json [2017-08-11 15:10:01.097] [TRACE] IsolatedTestRunnerAdapter - No captured browser, open http://

These error messages repeat for a while and then jumps straight to 99% with the remaining mutations marked as ‘survived’ mutants.

I created a simple angular service and corresponding Jasmine spec file to make sure i could isolate and reproduce the issue i am seeing:

stryker_test.service.js

/*global angular, require*/
require('angular');
(function() {
    'use strict';
    angular.module('stryker_testModule', [])
     .factory('stryker_test', function() {
        var index,
            dataValue = 0,
            testFunction = function() {
                for(index = 0; index < 10; index++){
                    dataValue += 1;
                }
            },

            getDataValue = function() {
                return dataValue;
            };

        return {
            testFunction: testFunction,
            getDataValue: getDataValue
        };
    });
}());

stryker_test.service.spec.js

/*global angular, require, describe, it, afterAll, beforeAll, beforeEach, afterEach, module, expect, inject, spyOn  */
require('angular-mocks');
(function() {
    'use strict';

    describe('stryker test service: ', function() {
        angular.mock.module.sharedInjector();
        beforeAll(angular.mock.module('stryker_testModule'));

        beforeAll(inject(function($injector) {
            this.strykerTestService = $injector.get('stryker_test');
        }));

        afterAll(angular.mock.module('stryker_testModule'));

        describe('When stryker test function is called', function() {

            beforeEach(function() {
                this.strykerTestService.testFunction();
            });

            it('should give the datavalue correct value', function() {
                expect(this.strykerTestService.getDataValue()).toEqual(10);
            });
        });
    });
}());

So far i am able to get around this issue by commenting out the code inside of applyMutations() in the UpdateOperatorMutator.js code:

stryker/src/mutators/UpdateOperatorMutator.js

"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
var esprima_1 = require("esprima");
var UpdateOperatorMutator = (function () {
    function UpdateOperatorMutator() {
        this.name = 'UpdateOperator';
        this.type = esprima_1.Syntax.UpdateExpression;
        this.operators = {
            '++': '--',
            '--': '++'
        };
    }
    UpdateOperatorMutator.prototype.applyMutations = function (node, copy) {
        //if (node.type === this.type && this.operators[node.operator]) {
        //    var mutatedNode = copy(node);
        //    mutatedNode.operator = this.operators[node.operator];
        //    return mutatedNode;
        //}
    };
    return UpdateOperatorMutator;
}());
exports.default = UpdateOperatorMutator;
//# sourceMappingURL=UpdateOperatorMutator.js.map

If it helps debug, i am using angular v1.4.7 and Jasmine v2.5.2. I’m not sure if this is a Stryker issue or a Karma issue.

Edit: I also saw this issue and thought it sounded kind of similar to what i am seeing. I’m not using karma-nightmare though.

Edit 2: Changing my for loop to use index+=1 instead of index++ also gets rid of the issue, but i’m pretty sure it’s just sneaking past Stryker so i never get into infinite loop scenarios during mutation testing.

[nicojs]

Hi Carlos, thanks for all the trouble. I think i found the issue.

TL;DR;

We should never let karma handle timeouts. As a work around: remove timeoutMs: 240000 from your stryker config will force Stryker to handle timeouts.

What’s wrong

The mutant is causing an infinite loop, which is not a problem normally for Stryker. It will kill the child process and spawn a new one, marking the mutant as a timeout. The problem is that not stryker, but Karma is reporting the timeout.

Karma itself has a mechanism of reporting timeouts when it hasn’t heard from the browser in x time. We assumed that the karma process would still be responsive afterwards, but it is not (never assume anything). I think we should fix this by forcing a timeout to be handled by Stryker.

[simondel]

As we haven’t heard anything about this so far, I’m closing this issue. If it’s still a problem, let us know!