Don't continue if the storybook throws an error

What

In order to avoid roadbumps we decided to ignore logged errors when evaluating your story list and getting a list of stories (sometimes people’s code logs errors but it’s not actually an issue).

We should re-evaluate this behaviour or add a flag to opt-into it.

Why

Often errors are logged when evaluating story files (if there’s a JS bug in one of the files). What this means (e.g this build at this url) is that we end up getting a truncated list of stories (all the ones that evaluated before the error was thrown).

Chromatic does not make you accepted deleted stories (which is a tricky problem actually) and this leads to a bad time.

However, some users have stuff logged to their error console as a matter of course and can’t necessarily remove it easily. So we still need a way to opt out.

How

1. Let’s change the default behaviour to cancel the build if there’s something logged to the error console.

2. Let’s add a flag --allow-console-errors that reverts to the current behaviour

3. In the case of 1., let’s say something like: “Your Storybook logged the following to the error console while loading. Errors loading your Storybook can lead to Chromatic getting your list of stories wrong and can cause a variety of problems. If you want to continue even with the errors, pass the --allow-console-errors flag to the chromatic command, however we do not recommend it.”

[tmeasday]

Also --allow-console-errors doesn’t appear to work.

[tmeasday]

Maybe I’m forgetful but I don’t remember us having this argument. I’m not seeing any reference to it in the entire repo.

It’s at the top of this ticket – we were going to add this flag to allow continuing the build even if the storybook outputs errors.

---

The current behaviour isn’t right btw – what I posted here is a bug.