RuleTester deeply merges default properties, including arrays (meaning specific elements are replaced, rather than the whole array)

Tell us about your environment

Environment Info:

Node version: v14.16.0 npm version: v7.14.0 Local ESLint version: v7.29.0 (Currently used) Global ESLint version: Not found Operating System: linux 4.4.0-19041-Microsoft

What parser (default, @babel/eslint-parser, @typescript-eslint/parser, etc.) are you using?

Default

Please show your full configuration:

Configuration

What did you do? Please include the actual source code causing the issue, as well as the command that you used to run ESLint.

const { RuleTester } = require('eslint');

const TestRule = {
  name: __filename,
  create(context) {
    console.log(context.settings);

    return {};
  },
};

const defaultSettings = {
  hookNames: ['beforeAll', 'beforeEach', 'afterAll', 'afterEach'],
};

const ruleTester = new RuleTester({ settings: defaultSettings });

ruleTester.run('my-rule', TestRule, {
  valid: [{ code: '', settings: { hookNames: ['beforeEachTest'] } }],
  invalid: [],
});

node file.js

What did you expect to happen?

For the settings property provided in the test to ~completely~ override the default set via the RuleTester config constructor parameter; in particular, I’d expect it to not merge arrays.

While I imagine this could be useful in some situations (specifically with env maybe?), the docs don’t mention this as being the behavior, and explicitly refer to the constructor config as used for default values:

The RuleTester constructor accepts an optional object argument, which can be used to specify defaults for your test cases

To me this makes it very surprising behaviour to merge arrays instead of override.

What actually happened? Please copy-paste the actual, raw output from ESLint.

The two hookNames arrays are merged, resulting in the first element in the default array being replaced by the first element in the test-level version of the array:

console.log
  {
    describeBlocks: [],
    hookNames: [ 'beforeEachTest', 'beforeEach', 'afterAll', 'afterEach' ]
  }

  at getOption (src/settings.ts:25:11)
      at Array.forEach (<anonymous>)

Steps to reproduce this issue:

See “What did you do?” section above. Steps are:

1. npm init -y && npm i -D eslint
2. Copy code into file.js
3. node file.js

Output will be:

{
  hookNames: [ 'beforeEachTest', 'beforeEach', 'afterAll', 'afterEach' ]
}

Are you willing to submit a pull request to fix this bug?

Yup 😃

[nzakas]

@G-Rath please add a comment on https://github.com/eslint/eslint/issues/13481. I won’t be revisiting the new config system until after ESLint v8.0.0 is out, but if you leave a comment there I’ll be sure to look into it more. I think we likely won’t know how big of a deal this is until we have a prototype to play with.

[mdjermanovic]

Closing since this works as intended with the current config system, and the discussion about the new config system will be continued in https://github.com/eslint/eslint/issues/13481.