[Bug] Bugs blocking usage of Berry

Describe the bug

I am trying to use Berry on existing projects. I’m running into multiple blockers, and some UX issues. I tried to look at using Berry with a beginner-mindset, blending out any sort of knowledge I may have about existing projects or dependency management. This bug report was filed to help identify a few ways in which Berry could help beginners successfully migrate to Berry.

• UX: After running Berry it reported that it “Failed with errors”. It was unclear what to do next, and whether it was a legitimate error or not. I had to open a log file in a temporary folder to see that a module could not be resolved (triggered through running postinstall after yarn?). Since this is caused through my usage of Berry, and wouldn’t happen with Yarn 1 or npm, I believe it could be made clearer what the next steps should be.
• UX: On first-run, Berry spams my terminal with one message per package that needs to be downloaded. For a large project, that can be tens of pages of terminal windows.
• Bug: After trying to run yarn jest to run tests, Berry couldn’t resolve modules across the workspace and failed with an unhandled promise rejection. I was left in a state where I’m unable to know how to continue.
• Bug: Trying to run yarn policies set-version <anything> when Berry is active resulted in an error Couldn't find a script named "policies".. I am unclear how to revert to Yarn 1.
• Perf: the link step for repeated invocations of Berry takes multiple seconds, as opposed to Yarn 1 which is immediate.
• Behavior change: It appears Berry does either not print the output of postinstall steps or doesn’t run them on repeated invocations (Tested on the Metro repo which builds all files on every invocation of Yarn). Is this expected?
• UX: Berry does not print anything to the terminal when running a command, which feels like something got stuck when running a slow command (like eslint for example).
• UX: I tried to run yarn policies set-version berry in a project with an older version of Yarn (1.17). The policies command was not found, but it was unclear that this was a command only added recently. I don’t know what could be done on existing versions but at least documentation could be clearer about updating Yarn 1 to the latest
• UX (subjective): Every line printed by Berry has a vertical bar (|) which looks visually jarring at times.
• UX (subjective): Every line contains > YN0 which feels repetitive.

Feel free to split these out into separate issues. My intent is to document the complete flow I went through and the issues that came up. I also tried Yarn on another smaller and older project. Because many of the dependencies were using versions that were old I could not get Berry to work. It may help to point out in the migration guide that all dependencies should be updated to the latest versions before changing dependency managers. It would be amazing if Berry could fail on install and tell me which versions of packages are known to work with Berry (ie. if I’m using an old version of resolve, Berry could automatically tell me which version is compatible).

To Reproduce

git clone [e-mail]:facebook/jest.git
cd jest
yarn policies set-version berry
yarn

Screenshots

Environment if relevant (please complete the following information):

• OS: macOS
• Node version: v12.8.0
• Yarn version: berry

[arcanis]

Hey, thanks for that 🙂

Bug: Trying to run yarn policies set-version <anything> when Berry is active resulted in an error Couldn’t find a script named “policies”… I am unclear how to revert to Yarn 1.

Related to #472

Perf: the link step for repeated invocations of Berry takes multiple seconds, as opposed to Yarn 1 which is immediate.

1.x was “cheating” by just not running at all if the .yarn-metadata had an older mtime than the package.json and yarn.lock files. It’s a fine approach, but here we follow a slightly different one where we want to run the full code path and make it fast (not the source, but this tweets sums it up perfectly: https://twitter.com/jordwalke/status/959018527091904512).

I find this approach important because the bypass had side effects that made the execution slightly unpredictable from the perspective of tooling (cf next point). For example as another data point, running yarn install here will always ensure that your cache is sound instead of just passing if one mtime happens to be the right one.

Behavior change: It appears Berry does either not print the output of postinstall steps or doesn’t run them on repeated invocations (Tested on the Metro repo which builds all files on every invocation of Yarn). Is this expected?

The 2.x will not print the logs for successful builds (just like 1.x), but it will also not run the postinstall scripts unless the dependency tree changed. The 1.x tried to have a similar behaviour (for example the bypass previously mentioned caused the postinstall scripts to not be run on every install) but was fairly buggy and sometimes unpredictable in this aspect.

UX: I tried to run yarn policies set-version berry in a project with an older version of Yarn (1.17). The policies command was not found, but it was unclear that this was a command only added recently. I don’t know what could be done on existing versions but at least documentation could be clearer about updating Yarn 1 to the latest

Yep, we should mention upgrading 1.x before installing 2.x 👍

It would be amazing if Berry could fail on install and tell me which versions of packages are known to work with Berry (ie. if I’m using an old version of resolve, Berry could automatically tell me which version is compatible).

That’s something we consider. The plugin-compat is here for this kind of purpose (for example it automatically adds the missing package extensions we’re aware of), and we’ll keep improving it based on similar feedback.

UX: On first-run, Berry spams my terminal with one message per package that needs to be downloaded. For a large project, that can be tens of pages of terminal windows.

It won’t work for first-time users, but for regular use we’ll implement #816 which will make it possible to filter out the messages you don’t want to see. By default I find the full list valuable though - the typical progress bars used to make it difficult to know what was causing hangs, especially when people shared screenshots.

(fails on yarn start with Failed to start Docusaurus server: Error: EROFS: read-only filesystem, open ‘/node_modules/docusaurus/lib/core/metadata.js’ – unclear how to proceed)

We try to throw the regular filesystem errors as much as possible when working with zip archives, hence the EROFS (Error Read-Only FS). I’ll add it to the troubleshooting though (basically it means that Docusaurus is writing within its own package folder instead of something like /node_modules/.cache/docusaurus).

[arcanis]

We were thinking with @gaetanmaisse that maybe the Yarn org could provide an helper package to store package artifacts. Maybe that would be good opportunity to mutualize efforts?