[Bug?]: `ENOENT` in Redwood contributing workflow using Yarn

Self-service

• I’d be willing to implement a fix

Describe the bug

Hey everyone. Opening this because I’m seeing a YN0001 error code, but I’m not entirely convinced this is a bug in Yarn yet.

I work on Redwood; there’s an issue in our contributing workflow that’s slowing down development. I’ll write it up as best I can below, but here’s a link to a video of the bug that I recommend watching if you can: https://s.tape.sh/9hTynWoS.

Both the Redwood framework and Redwood projects use Yarn 3, but they use the "node-modules" nodeLinker because we don’t follow the new rules yet.

Right now, to test new features in the framework on projects, we copy the framework’s packages’ dist to projects node_modules. So we’re modifying node_modules from under Yarn. (Note that we’re only doing this because we couldn’t seem to get yarn link to work, maybe because we don’t follow the new rules yet?)

We made our own contributing tool to do this, and while it mostly works quite well, if you try to yarn install after node_modules has been modified by our contributing tool, Yarn throws a YN0001 error code during the link step:

➤ YN0000: ┌ Link step
➤ YN0001: │ Error: ENOENT: no such file or directory, lstat '/Users/dom/prjcts/redwood_project/redwood_project/node_modules/@redwoodjs/core/node_modules/semver'
➤ YN0000: └ Completed in 0s 636ms

Is this actually a bug in yarn? Or are we breaking the contract by modifying node_modules and expecting yarn installs to still work after?

Lastly, thanks again for when we migrated from yarn 1 to yarn 3—everyone was amazingly helpful!

To reproduce

I’ll try to get this working in Sherlock, but in the meantime the video I linked to above shows the exact steps I took: https://s.tape.sh/9hTynWoS. And the issue should be reproducible locally by following the steps I took:

# Clone the Redwood framework, install deps and build
git clone https://github.com/redwoodjs/redwood.git
cd redwood
yarn
yarn build

# Make a redwood project
cd ~
yarn create redwood-app redwood-project

# Contributing workflow; `RWFW_PATH` is the path to your local copy of the framework
cd redwood-project
RWFW_PATH=~/redwood yarn rwfw project:deps && yarn
RWFW_PATH=~/redwood yarn rwfw project:copy

# Now installing again should throw the error
yarn
...
➤ YN0000: ┌ Link step
➤ YN0001: │ Error: ENOENT: no such file or directory, lstat '/Users/dom/prjcts/redwood_project/redwood_project/node_modules/@redwoodjs/core/node_modules/semver'
➤ YN0000: └ Completed in 0s 636ms

Environment

System:
    OS: macOS 12.1
    CPU: (8) arm64 Apple M1
  Binaries:
    Node: 16.15.0 - /private/var/folders/dt/yks4v5m53k114qxgz6jh4pgw0000gn/T/xfs-e9f6d41d/node
    Yarn: 3.2.0 - /private/var/folders/dt/yks4v5m53k114qxgz6jh4pgw0000gn/T/xfs-e9f6d41d/yarn
    npm: 8.5.5 - ~/.nvm/versions/node/v16.15.0/bin/npm
  npmPackages:
    jest: 27.5.1 => 27.5.1

Additional context

No response

[merceyz]

I see you noticed but for any future readers; the fix was backported and released in Yarn v3.2.1.

maybe the best way forward is for us to start migrating off the “node-modules” nodeLinker following these two guides?

Maybe, though you might be able to solve the symlink issue with --preserve-symlinks

[jtoar]

Maybe, though you might be able to solve the symlink issue with –preserve-symlinks

I’ll give that a shot—thanks for all the help on this one!