Proposal for loading plugins relative to the configs that depend on them

(Previous discussions include: #3458, #10125)

Background/Problem description

Currently, ESLint plugins and shareable configs are loaded from the location of the ESLint package, rather than the location of the config file where the plugins and configs are referenced. This leads to several problems:

• The current behavior requires end users to manually install any plugins required by a shareable config. As a result, a shareable config can’t add a plugin without requiring new manual installation steps from the end user. This greatly degrades the ergonomics of using custom rules from plugins in shareable configs, and results in increased pressure to add new rules and options to ESLint core.
• The current behavior assumes that if a user installs a config/plugin and ESLint in the same project, then ESLint will be able to load that config/plugin. This relies on an implementation detail of how npm works rather than a specified behavior, which leads to problems when using other package management strategies, e.g. with lerna. (More details about this problem can be found in #10125.)
• The current behavior leads to a large amount of confusion from users where ESLint behaves differently depending on whether it’s installed “globally” with npm.

To address these issues, many users have proposed loading all configs and plugins from the location of the config that depends on them (e.g. see #3458). For example, if a config has a field like plugins: react, ESLint would load eslint-plugin-react from the location of the config file itself. This would be beneficial because it would allow the author of a shareable config to control its dependencies, rather than the end user. It would also have a side-effect of removing the behavior change between local and global ESLint installations, which is a frequent source of confusion for users.

Unfortunately, implementing this scheme as-is would cause naming ambiguity. There could be two shareable configs which have dependencies on two different plugins that both happen to be called react (or they depend on two different versions of a plugin called react). If the end user depended on these two shareable configs and also configured a rule like react/some-rule in their top-level config, the end user’s config would be ambiguous because it wouldn’t be clear which react plugin they were referring to. Since the configurations for a given rule might be incompatible across different versions of a plugin, this could make it impossible to set the configuration for a particular rule or to override a configuration which was set by a shareable config. This would be an unacceptably poor user experience.

Another way to state the problem is that ESLint’s mechanism for naming rules in a config file (plugin-name/rule-name) is fundamentally unable to disambiguate two plugins that have the same name. Currently, there are no naming conflicts because all plugins are loaded from the location of the eslint package, so plugins effectively live in a global namespace. If plugins could be loaded as dependencies of shareable configs, then naming conflicts would start to become a problem. As a result, we need to be able to support having two plugins with the same name before we can support having plugins as dependencies of configs.

Design goals of solution

• A config author should be able to add or upgrade any plugin in their config, without requiring additional installation steps from end users that extend that config.
• The end user should maintain the ability to override any configuration setting inherited from an extended config.
• A config author should be able to extend any two other configs at the same time, and have ESLint lint their code successfully. (The two configs might advocate mutually-incompatible code styles, but that issue is out of scope for this proposal.)
• ESLint’s config-loading behavior should be compatible with the use of any package manager that follows the de-facto package.json spec, without relying on the implementation details of any particular package manager.
• Standalone config files should continue to be usable as shareable configs, and vice versa, without any changes.
• Shareable configs which currently have plugins as peerDependencies should be able to transition to the new solution without requiring changes to the configs of their users (or at least the vast majority of their users).
• The vast majority of existing configs in local-installation setups should continue to work with the new solution.

Summary of proposed solution

Since the problem stems from a limitation of ESLint’s naming scheme, I think the most straightforward solution to the problem would involve changing the way that rules are named in a config file. Specifically, this solution proposes using hierarchical naming for rules, where rules would be configured with something that looks like a path. (In other words, a user could refer to a rule like foo::react/some-rule for the version of eslint-plugin-react used by eslint-config-foo, and this would be a different rule than bar::react/some-rule, which would refer to the version of eslint-plugin-react used by eslint-config-bar.)

This is similar to a few other proposals discussed in #3458, but I’ve done a significant amount of work since then on precisely describing the behavior, figuring out the edge cases, and thinking through alternatives.

Details of proposed solution

When describing how rule name resolution works in this proposal, it’s useful to think of a “config tree” representing the dependencies between shareable configs and plugins. The root node is the end user’s config, and each node has a set of named children representing the shareable configs that it extends and plugins that it depends on. Here’s an example tree:

In this example, the end user’s config extends eslint-config-foo and eslint-config-bar. eslint-config-bar extends eslint-config-baz. eslint-config-foo and eslint-config-baz both depend on versions of eslint-plugin-react (perhaps different versions, although this doesn’t matter as far as resolution is concerned). eslint-config-baz also depends on eslint-plugin-import.

Rule name resolution

• Each reference to a plugin rule in a config consists of three parts: a config scope (i.e. a list of configs), a plugin name, and a rule name.
  • For example, in an existing rule configuration like react/no-typos, the config scope is an empty list, the plugin name is react, and the rule name is no-typos. (In existing rule configurations, the config scope is always an empty list.)
  • In a rule configuration like foo::bar::react/no-typos, the config scope is ['foo', 'bar'], the plugin name is react, and the rule name is no-typos.
  • The syntax shown here for writing a config scope (which uses :: as a separator) is up for bikeshedding. For now, I would recommend focusing on the abstract idea of a (configScope, pluginName, ruleName) triple; the question of how best to syntactically represent that can be decided independently of the rest of the proposal.
• Each reference to a plugin rule is also implicitly associated with a config in the config tree. References that appear in a config file are associated with that config file. References outside of a config file (e.g. from the command line or inline config comments) are associated with the root of the config tree.

To resolve a (configScope, pluginName, ruleName) triple to a loaded rule, which is referenced in a config baseConfig:

• If configScope is non-empty, find the child config of baseConfig in the config tree which has a name of configScope[0], and recursively resolve the rule (configScope.slice(1), pluginName, ruleName) from that config.
  • (If there is no such child config, the rule reference is invalid. ESLint should exit with a useful error message.)
• Otherwise, if configScope is empty:
  • If baseConfig has a direct child plugin with the name pluginName, or baseConfig is a plugin config from a plugin called pluginName, return the rule called ruleName from that plugin.
  • Otherwise, search for all plugins that are descendants of baseConfig in the config tree and have a name of pluginName.
    • If there is exactly one such plugin, return the rule called ruleName from that plugin.
    • If there are no such plugins, the rule reference is invalid. ESLint should exit with a useful error message.
    • If there is more than one such plugin, the rule reference is ambiguous. ESLint should exit with useful error message.
      • For example, this error message could include all of the matching plugins that were found, and provide a replacement rule reference that would disambiguate each of them. The user could the choose one of the replacements and copy-paste it into their config. This would make it simple for the user to resolve an ambiguity.

A few examples of this config resolution strategy, with the config tree given above (reproduced below for convenience):

Example config tree (same as above)

• If the end user’s config references the rule react/no-typos, the config scope is empty. Since the root node of the tree has multiple descendants called eslint-plugin-react, the rule reference is ambiguous.
• If the end user’s config references the rule bar::react/no-typos, the config scope is non-empty, so the resolution strategy then tries to resolve the rule react/no-typos from the eslint-config-bar node in the tree. Since there is only one descendent of that node called eslint-plugin-react, the rule would successfully resolve to the no-typos rule of that plugin.

Notable advantages and disadvantages of this strategy

• This strategy allows shareable configs to specify plugins and other shareable configs as direct dependencies, without manual installation steps by the user. It also ensures that the end user can always override any extended configuration.
• Any ambiguity in a rule reference in a given config file will be immediately apparent to the author of that config file, since the presence of an ambiguity only depends on the descendants of that config in the config tree. In other words, there is no situation where a particular user’s configuration would be broken and they would need to lobby the author of their shareable config to make a change (because in that case, the shareable config would be broken for all of its users and likely would have been fixed before publishing).
• The strategy is mostly backwards-compatible with existing setups, because in existing setups there is always at most one version of a plugin reachable from anywhere in a config tree. The exceptions are cases where an existing config unnecessarily uses a plugins array to override rule configurations from a shareable config’s plugin; with this change, the shareable config and the end user’s config would end up configuring two independent versions of the same plugin. To fix this in both versions, the end user’s config could simply remove that plugin from their plugins array. There are also probably setups now where a shareable config overrides the plugin rules configured by a sibling shareable config; these could be fixed in both versions by making one shareable config a child of the other.
• Adding a plugin to a shareable config is a breaking change for the shareable config, because it creates the possibility of an ambiguity in ancestor configs. I consider this to be acceptable because:
  • Adding a plugin to a shareable config would usually be a breaking change anyway, because the shareable config would be enabling new rules from that plugin, causing more errors to be reported to the end user.
  • With the current status quo, adding a plugin to a shareable config is always a breaking change because the end user needs to install it.
• With this strategy, an end user would be exposed to some details of the layout of shareable configs that they depend on. For example, if a shareable config eslint-config-foo has two descendant plugins with the same name, then the config scope that is needed to refer to those rules is “contagious”. In other words, if eslint-config-foo needs to use a reference like bar::react/no-typos to avoid ambiguities, then a config that extends eslint-config-foo needs to use something like foo::bar::react/no-typos to configure that rule.
  • I consider this to be acceptable because end users are already exposed to many of the details of their shareable configs, in that changes to shareable configs will lead to different linting errors/warnings being reported on the end user’s code. I think it’s very important to protect configs from caring about details of sibling shareable configs, since this would create implicit dependencies between the sibling configs where no dependency otherwise existed. However, I think it’s fine if configs are exposed to some details of descendent shareable configs, since there is already a dependency relationship between them anyway.
  • I believe that this problem (exposure to the details of dependency configs) is inherent to any solution that both (a) gives users the power to arbitrarily override third-party configuration, and (b) allows third-party configuration to pull in custom rules from multiple external sources. To illustrate this point, I’ll examine two other proposals that attempted to solve the problem, and argue that the problem still exists even with those alternate proposals.

    • https://github.com/eslint/eslint/issues/3458#issuecomment-255487235 proposed adding an exportedPlugins field for shareable configs. This has an advantage that it makes the exposed API of a shareable config explicit. However, with this alternate proposal, ESLint would have to require each shareable config to export all of its dependency configs (otherwise the end user would be unable to reference rules from the dependency configs in order to override them). As a result, the dependency chain of a shareable config would still end up in an end-user config.

    • https://github.com/eslint/eslint/issues/3458#issuecomment-257161846 proposed solving the problem by using plugins that depend on other plugins and reexport the rules of their dependencies, without any changes to ESLint core. It suggests two possible ways of re-exporting the rules: either a plugin could export them directly with the same name, or it could give the names a common prefix. Unfortunately, both of these strategies have problems:

      • If plugins re-export rules using the same names as the ones provided by their dependencies, then they will encounter a conflict when two of their dependencies export a rule with the same name. In effect, I think this strategy just changes the initial problem of having plugins with the same name to a new problem of having rules with the same name, and shifts the burden of solving it onto plugin authors. Even if a plugin created an ad-hoc fix for this by using a different name for one of the rules, it would still have the possibility of breaking again if one of its dependency plugins added a rule with the same name as one of its other dependency plugins.
      • If plugins put rules behind namespaces, then when a namespacing plugin depends on another namespacing plugin, the namespaces would end up getting compounded. In order to use one of the rules, an end user would need to repeat back multiple levels of namespaces, effectively restating the plugin’s dependency chain again. This would result in the same problem as with the current proposal.

      If these problems with alternative proposals seem like they would only occur in unlikely scenarios, I think it’s worth noting that the problem in this proposal of having deep conflicting rule names in this proposal would also be somewhat unlikely. We should certainly make sure users have the ability to handle those scenarios, and give straightforward advice about how to do so (namely, that shareable configs need to bump the major version when adding plugins or adding dependencies that add plugins), but it’s also important to make sure we make fair comparisons between proposals, and not compare the common cases of one proposal to the inconvenient edge cases of another.

---

I’ve spent a lot of time on this proposal, and would appreciate any feedback (particularly feedback that identifies problems with it). I think the current state of how plugins and configs are loaded is causing a large number of problems in the ecosystem, and I’m hoping we can work on this to finally pin down a solution.

[nzakas]

Hi everyone,

The ESLint team has just created an RFC process for complicated changes that require designs. It seems like this discussion is now split amongst at least three different issues and would benefit from being consolidated into an RFC:

https://github.com/eslint/rfcs/

I’d like to suggest that whomever is most interested (@not-an-aardvark?) please consolidate everything into an RFC proposal and close the outstanding issues so we can focus the conversation.

Thanks!

[not-an-aardvark]

If the top-level application lists all of the plugins, they will fall into the fallback, yep. It goes against the spirit of create-react-app & similar other scaffolding tools, though

Understood. I was just trying to determine how much of the ecosystem is currently affected by this issue.

@ljharb The issue is that ESLint’s plugin-loading depends on eslint being able to load a package that eslint doesn’t specify anywhere in package.json. The resulting system works in some situations due to package manager implementation details, but is unsound.

Can we move this discussion to https://github.com/eslint/eslint/issues/10125? I think that issue has a better description of the problem, whereas this issue is more focused on a particular proposal.