Make RuleTester test runner agnostic

Hey guys!

I have a few ESLint plugin repos I’m maintaining, where we use AVA as the test runner instead of mocha (including eslint-plugin-ava 😄). What we did until now was to wrap the whole RuleTester call in one AVA test, which made the test feedback pretty bad (it’s all good or all fail, you don’t know which one crashed, and the error messages were simplistic), just like if you would simply run the test file using node instead of mocha.

I noticed that by overriding the RuleTester’s before and itmethods, I could create proper test suite à la mocha, and created https://github.com/jfmengels/eslint-ava-rule-tester. It feels a bit hacky and fragile, so I would have liked to know if there was a better way, or if we could introduce one, to make RuleTester more test runner agnostic.

I provided screenshots of actual behavior with AVA and the behavior with eslint-ava-rule-tester in this PR (source code available there if needed) https://github.com/avajs/eslint-plugin-ava/pull/105.

I’m not sure about the solution to offer, but it would probably be something like adding a new parameter to RuleTester, which could be either an object with keys describe and it (which would be functions that create tests), or a new function that returns that.

var test = require('ava');

var avaTestCreator = {
  describe: function(test, method) {
    return method.apply(this);
  },
  it: function (text, method) {
    test(text, method);
  };
};

var ruleTester = new RuleTester({
    env: {
        es6: true
    } // and other options
}, avaTestCreator);

(cc @sindresorhus @twada @jamestalmage)

Oh, and by the way: Love the work that you guys are doing, thanks so much for the work you’re putting into this ❤️, I don’t express it often enough.

[nzakas]

Yeah, it and describe are part of the API. We should probably document them specifically.

@jamestalmage if you want to propose something around assert, please open a new issue.

[nzakas]

@jfmengels yeah, that’s a good spot. Probably just a section for customizing RuleTester there would do.