[Bug] Infinite loop / OOM when using yarn 2 pnp on a larger snowpack CRA codebase

• I’d be willing to implement a fix

Describe the bug We’re trying to adopt snowpack for our CRA codebase, but we have an issue that blocks us from using it.

What’s happening is that we’re running into a weird issue when using yarn 2 pnp with snowpack to build our frontend - it just runs in circles (probably some infinite loop or recursion) until it hits OOM.

To Reproduce

It’s triggered by simply trying to build something with a large number of dependencies:

import "@date-io/date-fns";
import "@date-io/moment";
import "@material-ui/core";
import "@material-ui/icons";
import "@material-ui/lab";
import "@material-ui/pickers";
import "@material-ui/styles";
import "@redux-saga/core";
import "@rooks/use-sessionstorage";
import "@storybook/addon-actions";
import "@storybook/addon-knobs";
import "@storybook/react";
import "array-move";
import "axios";
import "axios-mock-adapter";
import "classnames";
import "core-js";
import "create-react-class";
import "css-mediaquery";
import "d3";
import "d3-format";
import "d3-scale";
import "d3-selection";
import "d3-shape";
import "date-fns";
import "downshift";
import "enzyme";
import "enzyme-adapter-react-16";
import "enzyme-to-json";
import "eventsource";
import "file-saver";
import "formik";
import "fuzzaldrin-plus";
import "history";
import "html2canvas";
import "humanize-string";
import "i18next";
import "i18next-xhr-backend";
import "immutability-helper";
import "javascript-lp-solver";
import "javascript-time-ago";
import "jsonwebtoken";
import "jss";
import "jss-keyframes";

If you run snowpack build with yarn 1 or npm, the build works fine: https://github.com/alubbe/snowpack-yarn2/tree/a0c41b6333565858c857cce87889c014d0c06518 (yarn && yarn build)

If you run snowpack build with yarn 2, the build runs forever, then OOMs: https://github.com/alubbe/snowpack-yarn2/tree/4fec8a1171c2a05ba6f6656db56da0ea2ecf185d (yarn && yarn build)

But funnily enough, just comment out a few import lines, and suddenly it works, even with yarn 2. It works regardless of what I set max-heap-size/max-old-space-size to and regardless of what import statements I comment out (so this is not caused by a particular dependency). It seems to be purely caused by the size of the imported dependencies, which is really bizarre.

So for example, with yarn 2, if you comment out

// import "jsonwebtoken";
// import "jss";
// import "jss-keyframes";

it all works. So one might think one or more of those dependencies are causing the issue, but if you put them back in and instead comment out everything else, it also builds. It’s really confusing…

I’m not sure whether it’s a yarn bug or a snowpack bug, so I’ve also created a bug report on their tracker: https://www.pika.dev/npm/snowpack/discuss/518

Any help or pointers would be greatly appreciated!

Environment if relevant (please complete the following information):

• OS: Mac
• Node version: 14.5.0
• Yarn version: 2.1.1

Additional context

It’s the last issue that blocks us from adopting yarn 2 pnp in our large corporate monorepo - it would be fantastic to resolve it 😃

[FredKSchott]

Thanks for pinging me, happy to help debug this! No reason for us to be spinning up 1.3 GB of RAM, with or without yarn 2 😃 (PS: our discussion board is now here: https://github.com/pikapkg/snowpack/discussions, sorry I missed your original message)

Can you confirm that you haven’t set installOptions.sourceMap to true in your Snowpack config file? Can you set it to false explicitly, and check if that improves things?

The other thing worth trying, which would involve modifying the Snowpack package code a bit before you run it, would be to comment out the call to packageBundle.write(outputOptions) and see if you still hit the OOM. You could also try addingsourcemapExcludeSources: true to outputOptions. Rollup docs mention this as a way to make sourcemaps significantly smaller, which could also help.

If none of that helps, I’ll spin up an issue on our repo for someone to take a deeper look on our end. As far as I know, we’re not doing anything special with Rollup that would cause source maps to hang around longer than expected.

[alubbe]

woah - but if I compile yarn 2 from sources with your recent PR, it works! (see https://github.com/alubbe/snowpack-yarn2/commit/0f0ce6dd24cc3cae730fc3940512a0779c8a7d0e) I will try it out on our real codebase next week 😃