Feature request: Support snapshot testing

We test rules with a custom rule tester doing a snapshot test.

The built-in RuleTester can’t customize the output assertion, we don’t have the output to send to RuleTester, so we have to use SourceCodeFixer to apply fix from messages. Full implementation

We definitely want a better way to do snapshot test, maybe pass an assert option in future, but before that, can we expose SourceCodeFixer, so I can apply fix from the message?

The version of ESLint you are using.

8.0.0-beta.0

The problem you want to solve.

Fix our rule tester.

Are you willing to submit a pull request to implement this change?

Yes.

[bmish]

FYI: I am hoping to work on this / write an RFC in the near future.

We have snapshot testing in a related linter I work on, ember-template-lint, which could be useful as prior art:

• https://github.com/ember-template-lint/ember-template-lint/pull/1174
• https://github.com/ember-template-lint/ember-template-lint/pull/2150

[fisker]

Thanks for working on this.

I’m afraid it won’t work for my case, as I understand, the outputAssert used to assert output and messages[index].suggestions[index].output, but I need assert message too.

As you can see in this snapshot, there is a code frame pointing the report range, this is the inital propose adding this custom tester, because it’s hard to test the report range using the start/end.

How about pass all the output and messages(including suggestion.ouput)?

ruleTester.run("foo", replaceProgramWith5Rule, {
    valid: [],
    invalid: [
        {
            code: "var foo = bar;",
            output: "5",
			errors: 3,
            assert(actual, scenario) {
                assert.strictEqual(actual.output, scenario.output, 'Message');
                // Or maybe we can use `this`, but not important
                // assert.strictEqual(actual.output, this.output, 'Message');
                assert.strictEqual(actual.messages.length, scenario.errors, 'Message');
            },
        }
    ]
});

Also, @lydell need assert in each invalid scenario to use toMatchInlineSnapshot, but I don’t need that, so can we also support put assertOutput on the root?

ruleTester.run("foo", replaceProgramWith5Rule, {
	assertOutput() {
		// ...
	},
    valid: [],
    invalid: [
        {
            code: "var foo = bar;",
            output: "5",
			errors: 3,
        }
    ]
});