Webpack5 refresh is slow, lazyCompilation produces "singleton client api" errors

Describe the bug

Storybook’s fast refresh is very slow relative to other frameworks, specifically Next.js, which also uses Webpack 5. In order to improve fast refresh/rebuild times i’ve enabled config.experiments = { lazyCompilation: true }; in main.js webpack config which is a significant improvement. Unfortunately, on story refresh, storybook throws the following errors from multiple addons:

client_api.js:35 Uncaught (in promise) Error: Singleton client API not yet initialized, cannot call addDecorator
    at addDecorator (client_api.js:35)
    at addDecorator.js-generated-other-entry.js:24
    at Array.forEach (<anonymous>)
    at addDecorator.js-generated-other-entry.js:23
    at Array.forEach (<anonymous>)
    at ../../node_modules/@storybook/addon-actions/dist/esm/preset/addDecorator.js-generated-other-entry.js

To Reproduce

• enable webpack 5. Compare fast refresh times in a large project to other frameworks like Next.
• enable lazyCompilation in main.js webpack config
• view a story and refresh the page

System

  System:
    OS: macOS 10.15.7
    CPU: (8) x64 Intel(R) Core(TM) i7-8569U CPU @ 2.80GHz
  Binaries:
    Node: 12.14.1 -
    Yarn: 1.22.10 -
    npm: 6.13.4 - 
  Browsers:
    Chrome: 90.0.4430.212
    Firefox: 87.0
    Safari: 13.1.3
  npmPackages:
    @storybook/addon-actions: 6.2.9 => 6.2.9 
    @storybook/addon-knobs: 6.2.9 => 6.2.9 

[tmeasday]

This is interesting; I wonder why lazy compilation would help at all given SB isn’t code split.

@dontsave do you have an example repo you can upload that we can look at?

Storybook isn’t currently structured in such a way that it can really take advantage of code splitting/lazy compilation. I have no idea what WP is doing if you include that flag without splitting, so I can’t speak to the specific error, but it isn’t surprising that there would be problems if code was being partially loaded/reloaded.

In 6.4 we plan to re-architecture the core runtime of SB to allow you to properly take advantage of such features. https://github.com/tmeasday/storybook-skeleton is a prototype that shows it in action.

PS – if you are seeing issues with unexplained rebuild times in SB but not other systems using WP over the same codebase, it could be due to SB’s use of require.context: https://github.com/webpack/webpack/issues/13636

I would definitely advise trying remove the stories glob or restrict it to apply to less folders/potential story files if your project contains a lot of files. This is something else we will be looking closely at over the coming weeks.

[kaelig]

Thank you for sharing, @vidal7 , I’m noticing a performance improvements in rebuilds using the workaround (in our cloud development environment, on a large codebase with ~500 stories, webpack 4, @storybook/react v6.3.7).

• Before: between 7s and 12s
• After: between 4.5s and 6s

I believe I tried this approach a while ago and reverted the change because it wasn’t convenient: the glob seems to be static, so adding/renaming story files requires a restart, whereas the desired behavior is dynamic.

❓ How to get around this? @vidal7 any pointers?

Other finding: with fast-glob, the initial start up seems to be a few seconds faster than with glob.

---

Other workaround: user-defined globs

Another workaround is to offer people a way to only load a subset of stories via the command line, like so:

$ STORYBOOK_STORIES=./src/components/Button/Button.stories.ts yarn storybook

Your main.js should look something like this (simplified for this example):

// Load a subset of stories, using a custom glob
//
// Fastest: single story
// $ STORYBOOK_STORIES=./src/components/Button/Button.stories.ts yarn storybook
//
// Fast: all stories in a specific component directory
// $ STORYBOOK_STORIES=./src/components/Button/**/*.stories.@(mdx|ts) yarn storybook

module.exports = {
  stories: process.env.STORYBOOK_STORIES
           ? [process.env.STORYBOOK_STORIES]
           : ['./src/**/*.stories.ts']
};