Remove specialness of sub-plugins

Summary

Make it so that the plugins config key in plugin options is not the only key that allows running gatsby-* in sub-plugins.

module.exports = {
  plugins: [{
    resolve: `some-plugin`,
    options: {
      // we're talking about this plugins key, not the top-level one above
      plugins: [...]
    }
  }]
}

Motivation

Continuation of: https://github.com/gatsbyjs/gatsby/issues/15486#issuecomment-515276649

Basically, sub-plugins are special in Gatsby, as seen below in load-plugins:

// Create a "flattened" array of plugins with all subplugins
// brought to the top-level. This simplifies running gatsby-* files
// for subplugins.
const flattenPlugins = plugins => {
  const flattened = []
  const extractPlugins = plugin => {
    plugin.pluginOptions.plugins.forEach(subPlugin => {
      flattened.push(subPlugin)
      extractPlugins(subPlugin)
    })
  }

  plugins.forEach(plugin => {
    flattened.push(plugin)
    extractPlugins(plugin)
  })

  return flattened
}

This presents a problem for plugins such as gatsby-plugin-mdx, which tries to support the existing ecosystem of gatsby-remark-plugins by using a gatsbyRemarkPlugins config key:

// gatsby-config
{
  resolve: `gatsby-plugin-mdx`,
  options: {
    gatsbyRemarkPlugins: [{
      resolve: `gatsby-remark-images`,
      options: {
        // It's important to specify the maxWidth (in pixels) of
        // the content container as this plugin uses this as the
        // base for generating different widths of each image.
        maxWidth: 590,
      },
    }]
  }
}

This is important because gatsby-plugin-mdx strives to make converting from remark as painless as possible and gatsbyRemarkPlugins isn’t handled the same way as plugins, making it a second-class experience that results in bugs.

gatsby-plugin-mdx also supports passthrough of @mdx-js/mdx core options which includes two plugin types: remarkPlugins and rehypePlugins. Looking forward to the future, I’d love to use this as a place to gradually evolve an API to unify the rehype, remark, and gatsby plugins ecosystems under the same plugin API. This would greatly reduce the complexity of gatsby-remark plugin support in gatsby-plugin-mdx and (hopefully) make it easier to work with the existing ecosystem of remark plugins and “enhanced” gatsby-remark plugins in the gatsby-remark sub-ecosystem. The issue here is required for that future work to be successful.

So, being an edge case, gatsby-plugin-mdx has 4 config keys where it makes sense to “activate” as gatsby plugins. Other plugins may have a more reasonable two config keys.

Basic example

User-side, code doesn’t change. The original bug described above would be fixed.

Internally, I’d imagine the change (conceptually) looks something like this, where plugins can define their own list of sub-keys that would be required into the plugins list for this. (I don’t know how we would do this automatically without additional config, so if someone knows that’d be cooool).

const extractPlugins = plugin => {
+  const { subPluginKeys } = require(plugin)
+  for (const plugins in subPluginKeys) {
      plugins.forEach(subPlugin => {
        flattened.push(subPlugin)
        extractPlugins(subPlugin)
    })
  }
}

cc/ @sidharthachatterjee @KyleAMathews who I’ve talked this through with a bit

[ChristopherBiscardi]

going to make this change in gatsby-config itself which will be a simpler implementation. adding gatsbyRemarkPlugins to root plugins array by default.

[rheinardkorf]

Ping! Keeping this non-stale as its still an issue.