Change Request: Invalid test cases that yield suggestions should require a `suggestions` assertion

ESLint version

8.0.0-rc.0

What problem do you want to solve?

• Currently, it’s easy to forget to assert what suggestions a test case yields, or that a test case yields no assertions. As a result, this can be an area of poor test coverage, leading to bugs around suggestions.
• It’s also inconvenient to have to add suggestions: [] to every single invalid test case for a rule or tests that do not produce suggestions, in order to achieve the desired test coverage.

What do you think is the correct solution?

Invalid test cases that yield suggestions should require a suggestions assertion.

This would be the same as how invalid test cases that produce an autofix require an output assertion, and fail with this error:

AssertionError [ERR_ASSERTION]: The rule fixed the code. Please add 'output' property.

This behavior was added as a breaking change in ESLint v7.

We should add a similar assertion for suggestions.

AssertionError [ERR_ASSERTION]: The rule violation produced a suggestion. Please add a 'suggestions' property to the error object.

Example of invalid test case with suggestions assertion:

new RuleTester().run('example-rule', rule, {
  valid: [],
  invalid: [
    {
      code: `foo();`,
      output: null,
      errors: [
        {
          messageId: 'bar',
          type: 'Property',
          suggestions: [], // or array of suggestions if rule provides them
        },
      ],
    },
  ]
});

Participation

• I am willing to submit a pull request for this change.

Additional comments

No response

[bmish]

I have opened an RFC for this issue, along with some related rule test validation improvements: https://github.com/eslint/rfcs/pull/84

[bmish]

Still planning to do this, assigned myself.