Change default rule schema to `schema: []`

The problem you want to solve.

Today, rule schemas (meta.schema) are optional. This has the following effects:

• If a rule does not specify a schema, nothing is enforced.
• Rules with options may not have a schema specified, despite it being best practice to specify a schema for rule options. This increases the chances of developers passing invalid rule options and encountering unexpected behavior / bugs in rules.
• If a rule author wants to enforce that no options are passed to their rule, they have to manually specify schema: [], otherwise it’s possible that developers could accidentally provide useless rule options to their rule without knowing it.

There is an existing, third-party lint rule eslint-plugin/require-meta-schema to enforce that rules have schemas, but this rule is optional and thus not widely used.

Your take on the correct solution to problem.

Change the default rule schema to schema: [] to enforce that no rule options are passed to rules that do not specify a schema.

This has the desirable consequence of requiring rules with options to begin specifying their schema if they did not already.

This would be a breaking change suitable for the next major release.

Making ESLint’s default behavior more strict like this goes along with many recent changes to tighten validation (i.e. more RuleTester class validation in ESLint 7, increased rule configuration validation in ESLint 6, etc) to improve rule usability and reduce the chance of mistakes.

And in addition to the obvious benefits of increased rule schema usage such as reduced chance of silent mistakes/bugs, being able to rely on having rule schemas available could enable:

• Improved/autogenerated documentation regarding rule options
• TypeScript-style autocomplete/IntelliSense when configuring a rule

Are you willing to submit a pull request to implement this change?

Yes. I am open to writing the RFC and implementation.

[bmish]

I put together a repository called eslint-ecosystem-analyzer with scripts to perform an analysis of the ESLint plugin ecosystem. It retrieves the top ESLint plugin repositories by searching GitHub and then clones them so that metrics can be computed.

The analysis output is included below. To summarize the findings:

• For the top 100 ESLint plugins, 1% of total rules have options but are missing a schema (or said another way, 5% of rules with options are missing a schema). The percentages are only slightly higher (worse) when the top 1,000 ESLint plugins are considered (the top 1,000 results include many repositories that haven’t been maintained nor used in years).
• For the top 100 ESLint plugins, 94% of rules are object rules. The percentage is only slightly lower (worse) when the top 1,000 ESLint plugins are considered.

These findings seem very favorable toward:

• Requiring rules with options to provide schemas in ESLint 9
• Removing support for function-based rules in ESLint 9

In order to move forward with both of these breaking changes:

1. Write an RFC for each
2. In a minor version of ESLint 8, add deprecation warnings when running tests on a rule that does not meet the new requirements
3. Utilize eslint-plugin-eslint-plugin to help rule authors prepare as mentioned in the above comment (https://github.com/eslint/eslint/issues/14709#issuecomment-861946833)

Let me know what you think of this analysis and plan. If it sounds good, I could start on RFCs.

╔══════════════════════════╤═════════════════════════╤═══════════════════════════════════════════════╤════════════════════════════════════════════════╤══════════════════════════╗
║ Metric                   │ Value (Top 100 Plugins) │ Value (Top 1000 Plugins, Updated Last 1 Year) │ Value (Top 1000 Plugins, Updated Last 2 Years) │ Value (Top 1000 Plugins) ║
╟──────────────────────────┼─────────────────────────┼───────────────────────────────────────────────┼────────────────────────────────────────────────┼──────────────────────────╢
║ Plugins Found            │ 100                     │ 639                                           │ 777                                            │ 1000                     ║
╟──────────────────────────┼─────────────────────────┼───────────────────────────────────────────────┼────────────────────────────────────────────────┼──────────────────────────╢
║ Plugins With Rules Found │ 79                      │ 446                                           │ 533                                            │ 677                      ║
╟──────────────────────────┼─────────────────────────┼───────────────────────────────────────────────┼────────────────────────────────────────────────┼──────────────────────────╢
║ Average Rules Per Plugin │ 22.09                   │ 7.96                                          │ 7.23                                           │ 6.27                     ║
╚══════════════════════════╧═════════════════════════╧═══════════════════════════════════════════════╧════════════════════════════════════════════════╧══════════════════════════╝

╔══════════════════════════════════════════════════════════════════╤═════════════════════╤═══════════════════════════════════════════╤════════════════════════════════════════════╤══════════════════════╗
║ Metric                                                           │ % (Top 100 Plugins) │ % (Top 1000 Plugins, Updated Last 1 Year) │ % (Top 1000 Plugins, Updated Last 2 Years) │ % (Top 1000 Plugins) ║
╟──────────────────────────────────────────────────────────────────┼─────────────────────┼───────────────────────────────────────────┼────────────────────────────────────────────┼──────────────────────╢
║ Rules With Options                                               │ 27.16               │ 27.84                                     │ 26.77                                      │ 26.46                ║
╟──────────────────────────────────────────────────────────────────┼─────────────────────┼───────────────────────────────────────────┼────────────────────────────────────────────┼──────────────────────╢
║ Rules With Options But Missing Schema, Out of Total Rules        │ 1.49                │ 1.91                                      │ 1.95                                       │ 2.26                 ║
╟──────────────────────────────────────────────────────────────────┼─────────────────────┼───────────────────────────────────────────┼────────────────────────────────────────────┼──────────────────────╢
║ Rules With Options But Missing Schema, Out of Rules With Options │ 5.49                │ 6.88                                      │ 7.27                                       │ 8.55                 ║
╚══════════════════════════════════════════════════════════════════╧═════════════════════╧═══════════════════════════════════════════╧════════════════════════════════════════════╧══════════════════════╝

╔═══════════════╤═════════════════════╤═══════════════════════════════════════════╤════════════════════════════════════════════╤══════════════════════╗
║ Rule Type     │ % (Top 100 Plugins) │ % (Top 1000 Plugins, Updated Last 1 Year) │ % (Top 1000 Plugins, Updated Last 2 Years) │ % (Top 1000 Plugins) ║
╟───────────────┼─────────────────────┼───────────────────────────────────────────┼────────────────────────────────────────────┼──────────────────────╢
║ Object Rule   │ 93.81               │ 91.41                                     │ 89.74                                      │ 86.80                ║
╟───────────────┼─────────────────────┼───────────────────────────────────────────┼────────────────────────────────────────────┼──────────────────────╢
║ Function Rule │ 4.01                │ 6.14                                      │ 7.84                                       │ 10.91                ║
╟───────────────┼─────────────────────┼───────────────────────────────────────────┼────────────────────────────────────────────┼──────────────────────╢
║ Unknown       │ 2.18                │ 2.45                                      │ 2.41                                       │ 2.29                 ║
╚═══════════════╧═════════════════════╧═══════════════════════════════════════════╧════════════════════════════════════════════╧══════════════════════╝

[btmills]

That is amazing 👏 I’m also supportive and feel much better knowing that the vast majority of the ecosystem is already prepared.