Proposal: hooks to inspect and modify report data

After running into some of the issues described in both #138 and #154, I have a suggestion that might make it easier for folks to work around the lack of built-in support for features like matrix builds (desktop + mobile) and URL determinism, taking the burden of implementing them off you and other contributors:

Lots of other platforms and tools (Rollup is one) offer declarative hooks that allow the configuration to inspect and even modify data (in place) at specific points in the execution lifecycle.

In my specific case, I’m trying to use Lighthouse CI to run scheduled reports on a list of URLs that either aren’t under our control or have content that isn’t managed in GitHub. Currently, builds for which the git SHA hasn’t changed (which will be typical because the actions are run on a schedule) fail because the hashes are the same.

I’m going to gloss very heavily over the implementation details because I haven’t looked at how builds are scheduled or structured, but hopefully you’ll get where I’m going. Some examples:

• Before running the builds, a hook could modify the list of builds to run to include desktop runs for each, e.g.

  module.exports = {
    ci: {
      hooks: {
        beforeRunBuilds ({ builds }) {
          for (const { hash, settings, ...build } of builds) {
            builds.push({
              ...build,
              hash: `${hash}-desktop`,
              settings: {
                ...settings,
                emulatedFormFactor: 'desktop'
              }
            })
          }
        }
      }
    }
  }
  

• Before the builds are uploaded, a hook could modify the hashes to ensure that they’re unique (in the case of my scheduled runs):

  const date = (new Date()).d.toLocaleDateString()
  module.exports = {
    ci: {
      hooks: {
        beforeUploadBuild ({ build }) {
          build.hash = `${build.hash}@${date}`
        }
      }
    }
  }
  

• Or there could even be a hash hook to modify the data used to generate the hash for each build:

  const date = (new Date()).d.toLocaleDateString()
  module.exports = {
    ci: {
      hooks: {
        beforeHash ({ hash }) {
          // assuming a hash object that's JSON.stringify()'d before being hashed into a string
          hash.date = date
        }
      }
    }
  }
  

If you’re uncomfortable with hooks living in the configuration, plugins might be a good place to enable them. That would also make it easier for folks to distribute reusable packages that do very specific things: enable desktop + mobile builds, or further expand a matrix of testing environments; ensure that build hashes involve the current date, time, or some other changing context.

LHCI is such a great tool, and I would love to help with this if I can. Thanks for considering it!

[patrickhulce]

That’s the rough area we’ll need for beforeCollect but unfortunately it’s the one that requires the most internal restructuring. (We’ll need to create the idea of a collection object that contains the URL, settings, etc which currently exists only implicitly)

2 and 3 might be easier to get started with contributing if you want to start there? 😃

[patrickhulce]

For the specific problem of duplicate hashes, yes but as a broader solution to the “use desktop settings for this run” problem I like the hooks 😃