Offer source level fine grained control of mutations and mutators

Is your feature request related to a problem? Please describe. Controlling mutations and mutators within the tested project source code for fine grained control. Proposed syntax looks like this

// Stryker disable once all : Mutation has been reviewed and cannot be killed

Somer users want to keep their mutation coverage at 100%. But Stryker.Net may generates mutation that cannot be killed due to some external dependencies or (user project) internal design choices. Users looking for 100% coverage want to be able to review those mutations and somehow mark them as to be ignored. From a practical perspective, the best place to flag those mutations as to be ignored is within the source code itself, using single line comments to give instructions

Describe the solution you’d like I also like the syntax. But it is somewhat different from the JS proposal (next which does not really makes to me). Assuming this difference an issue, I think I could implement this, here are the envisioned specs:

• works on whole statements at minimum: this is due to how Roslyn attaches comments and code. Will do finer grained control (e.g. expression) if possible, but not a priority
• works on unitary/automatic scope as a priority, manually defined scope (i.e using disable and restore) is a secondary objective. This derives from the assumption this feature should be used sparingly and not in broad strokes.
• mutations will still be generated but will be flagged as ignored. First implementations may simply skip generating mutations. Remark: skipping mutating altogether may provide a workaround for when a mutator triggers an exception blocking all usage of Stryker ⇒ we need to choose which behavior we want.
• works at statement, block or declaration level. The rule then apply to the attached scope. eg

{
  // Stryker disable once all
  int x = 2+2;
  int y = x+3;
}

⇒ disable mutation for int x = 2+2;

// Stryker disable once all
{
 int x = 2+2;
 int y = x+3;
}

⇒ disable mutation for both lines

• manual defined scope (i.e using disable and restore) does not work across scope (i.e. you cannot disable mutations in one method and reenable it somewhere in the next method). This is due to the tree parsing logic we use which makes this needlessly difficult to implement; plus it should be used sparingly
• full comment syntax should be // Stryker [disable[| once]|restore] [all|mutator1[|,mutator2...]][| : reason for disabling] the 'reason for disabling will be used to comment skipped mutations (assuming mutations are stil generated, see above) ⇒ we need to decide if we want to make this part mandatory or not. I am in favor of making it mandatory
• Stryker could still generates mutations in diff-mode when the lines in the scope were mutated, giving an opportunity to review those lines. Note this probably means significant effort, as it means accessing diff information during mutation phase. Probably a feature for a future version, if possible at all.
• Stryker could issue a warning, or block altogether if ratio of skipped mutants is above a user defined threshold. This may be useful for teams to control overall discipline with the feature.

Describe alternatives you’ve considered No better alternative identified. Potential candidates:

• keeping some mutation database along with the project: requires extra work + difficult to properly identify mutants across project version
• extending options syntax, such as providing the list of lines to be ignored (eg class.cs: 35-37, 52-56). Cumbersome to maintain as there is a chance of line numbers having to be changed following some refactoring. May push user to disable file entirely to avoid this maintenance burden
• restricting mutators to ensure no un killable mutants can be generated: may lead to needlessly limit mutators because they may generate undue survivors. Plus, there still would be situation were a mutant could logically be killed, but can’t for practical reason (e.g : providing a callback to some external library the user has no control over)

Additional context Check discussion #1503 for further context and opinions

[rouke-broersma]

I propose we simply disallow multiline comments and mixed inline comments. Only normal comments are allowed and they must be on their own line.

If someone really wants to disable part of a statement or expression mutation they can use the ugly example which works and is imo very clear in showing that something funky is going on there.

[psfinaki]

OK, you convinced me with not requiring a comment 😃

As for once vs next, I slightly tilt to once, it clearly means “one case”, whereas next could refer to some next lines. There is also “ad hoc” expression, but it adds a bit different vibe probably.