Support yarn 2's PnP natively in ngcc

🚀 feature request

Relevant Package

Mainly @angular/compiler-cli

Description

It is currently impossible to use the ivy renderer in a workspace managed by yarn 2 with the pnp linker, unless using yarn’s pnpify command.

There are two blocking issues and one that’s more of a nuisance, all of which can be tackled by angular.

The blockers:

• ngcc requires the node_modules folder to exist.
• ngcc uses a custom implementation of the node module resolution algorithm, which is not supported in PnP.

The nuisance:

• All angular packages that need to be run through ngcc have to be unplugged manually (e.g. yarn unplug @angular/core), as ngcc writes to these package folders.

I’ve created a proof-of-concept repository where @angular/compiler-cli supports PnP natively: https://github.com/bgotink/angular-pnp-ivy-poc

Describe the solution you’d like

Native support for pnp in ngcc. I don’t necessarily mean any code specific to pnp. A removal of the custom node_modules resolution algorithm and replacing it with calls to require.resolve with the paths option or via module.createRequire should work in all environments.

For the unplugging: the ideal solution (from yarn’s point of view) would be to not write to the package folders (see also #33395). Unplugging the packages works fine though, so this is a nuisance rather than a blocker. Angular could help alleviate this a bit by indicating to yarn that the relevant packages need to be unplugged using a "preferUnplugged: true flag in the package.json files. That way developers using angular don’t need to worry about unplugging, and if angular ever moves away from writing to the package folders, that flag can be removed and developers will automatically get an improved developer experience when they update. Of course this only applies to angular’s own packages. Third party angular libraries will still need to be unplugged (either by having their own package marked as preferUnplugged or by manually unplugging).

The @angular/compiler-cli package also needs to be unplugged because it tries to write a lockfile in its own location. If this could be moved to e.g. node_modules/.cache/ that’d make unplugging no longer necessary. That’s the same folder used by babel, terser, nyc and others to store their caches in, e.g. using find-cache-dir. Yarn 2 supports this behaviour by explicitly keeping node_modules/.cache folders around (the install command removes any other file or folder in node_modules).

Describe alternatives you’ve considered

• Yarn 2 supports patching installed packages. The repo mentioned in the description contains a patch for @angular/compiler-cli that replaces the custom node module resolution algorithm with calls to require.resolve. Of course it’s hardly ideal that packages are patched. This is more of a temporary solution to work around issues until they are patched upstream.
• Yarn has a pnpify package that, when wrapped around another command, will mock the node_modules folder. This works already, but it introduces overhead. The time difference in building the PoC repo: 2 seconds real time, from 9.84 to 11.85 seconds; 3 seconds user time, from 13.67 to 16.62 seconds. These times are the average of five builds each for yarn ng build and yarn pnpify ng build.

[petebacondarwin]

Yes, we are currently implementing the “linker” approach set out in the RFC. You can follow progress here https://github.com/orgs/angular/projects/2

[petebacondarwin]

@bgotink - thank you for this detailed and well thought out feature-request. And also for the proof-of-concept - this sort of thing is invaluable.

We are currently looking into the future of ngcc and where we go next to avoid some of the constraints that ngcc has. This may well involve a different strategy that would avoid having to modify published library code in the way that ngcc does right now. We will be publishing our ideas (probably in the form of an RFC) in the near future. So I think it would be better to consider this PR in the light of that, since I would rather focus our efforts on a long term solution rather than continuing to fix up ngcc.

Let’s come back to this issue in a few weeks when we are in a position to compare the cost of doing it with the other options on the table.