Simplify configuration syntax

Our current configuration syntax has the following issues:

1. Retrieving plugin outputs is problematic because plugin order conflicts with lifecycle order (see #186). This makes it very hard with the current design for plugins to communicate outputs/information to the user or to other plugins.
2. Users don’t understand when plugin methods are executed and in which order, because this is done implicitly/magically. This means they have to always run netlify build --dry. They should not have to use a separate command to understand what’s going on with their build.
3. Users will want some plugin methods to run at different times than what the plugins is doing by default. We currently fix this by letting each plugin define its own runsOn option, but this requires more work for both the plugin author and the user.

This is a proposal that would solve all those problems by drawing inspiration from GitHub actions design. It also re-uses their syntax so it’s already familiar to our users.

This solution keeps config.build.lifecycle.* so users can still run specific hooks during different lifecycles: post-deploy, pre-build, on errors, etc.

A plugin would look like this:

module.exports = {
  name: 'netlify-plugin-example',
  methods: {
    doThis({ pluginConfig: { foo } }) {
      console.log(`Done after build: ${foo}`)
      return { exampleUrl: 'url' }
    },
    doThat({ outputs: { exampleUrl } }) {
      console.log(`Output from previous step: ${exampleUrl}`)
      console.log('Done after deploy')
    },
  },
}

A configuration would look like this (notice the plugins property is not needed anymore):

build:
  lifecycle:
    build:
      - npm run build
      - uses: netlify-plugin-example#doThis
        with:
          foo: bar
    deploy: 
      - echo ${outputs:exampleUrl}
      - uses: netlify-plugin-example#doThat
      - uses: netlify-plugin-another
        with:
          url: ${outputs:exampleUrl} 

What do you think?

[biilmann]

Have a bunch of thoughts on this and it might be good to setup a shirt call to talk through. Here’s my initial super brief take since I’m at the A16Z Summit without a lot of time to write something longer.

I think we’re still not quite there with the configuration format, but from a product philosophy perspective I do think the current plugin syntax is closer to the mark than the Actions inspired version.

I think users should be able to add things like a lighthouse score plugin without having to touch lifecycles at all and still just use the current one step command: "yarn build" kind of syntax.

I could also easily imagine a future where plugins could be activated from the UI without touching the config file at all, and the separate plugin config makes more sense in that world in my opinion.

In my view we should do more to make the build steps and what happens in them obvious in the UI, rather than focusing on teaching our users to build more complex flows in configuration files.

I’ll need more time to dig into the current set of plugins to see if there’s opportunity to simplify the syntax more, but I think we should make sure that the syntax we choose works in a way where you can easily imagine an existing user adding a plug-in to their current build without restructuring the existing toml file to be based on life cycles.

[sdras]

For me, @ehmicky’s design proposal is actually cleaner. I’ve found

plugins:
  - type: netlify-plugin-one

to be a bit verbose without adding much value. Coming from working with actions, this unifies our configuration and will be a little easier for devs with that experience to follow. This piece is important- the likelihood that a dev would be exploring both actions and our build plugins is high. And as I mentioned in the very first call about Build Plugins, this ability to support GitHub actions is a force multiplier and allows us to work much more effectively in the ecosystem.

I’m also really interested in having the plugin return outputs for other plugins to consume. Funnily enough, this was also another point I brought up in the first meeting, though less from a “one returning another” standpoint and more from a “we could have an orchestrator like durable functions”, though one could argue that the configuration file is the orchestrator here.

All in all, I’m very much in favor of @ehmicky’s proposal! Great work.