Proposal for an alternative linking strategy

The current linking strategy does not play well with some frameworks and platforms. For example, assume I have an Angular app in repo A, and an Angular library in repo L. Let’s further assume that both A and L have an identical dependency D. If A and L were independent repos (in other words, not part of a Lerna mono-repo), then npm install would put one D under A for use by both A and L. On the other hand, if A and L are sub-repos in a lerna mono-repo, lerna bootstrap installs a symlink for L under A, which points to the actual L, which has its own D, which is used at run-time for calls from L to D This causes problems for various packages which care about the “identity” of some class for whatever reason.

For example, I just ran into this problem with rxjs. That library checks the arguments to combineLatest to check if they are instanceof Observable. However, definitions of Observable from distinct copies of the package, even if byte-for-byte identical, are not considered “the same”. An instance created with one of them will not be an instanceof the other one. The result is that an Observable created in the app A, when passed a utility in library L which uses combineLatest, will fail, because it is not considered an observable in the sense that L (L’s copy of rxjs) understands it.

Another example is Angular dependency injection, which again relies on matching types. Let’s say I have a component in my library L which injects ElementRef, a built-in Angular type. If I now try to use that component from A, the ElementRef in the library L will not be found in the world of available injectables, because that is governed by imports that I specified in the app A, whose copy of Angular is separate from that being used by the library.

To solve this problem, which by the way has been extensively discussed by a variety of frustrated people on various boards including that for Angular, and affects npm link in exactly the same way it affects lerna, I have resorted to patching TypeScript’s module resolution strategy by using the paths compiler option in tsconfig.json. This option allows me to map an npm package path such as @angular/core to a specific directory inside my project (app A) directory, so in the entire process of resolving node references, including those made from compiled sources within library L, the @angular/core under L is ignored in deference to that under A. Other people have reported some success using the --preserve-symlinks option to various angular-cli building commands.

Assuming the following dependency structure:

repo
| -- A
|    | -- D
|    | -- L
| -- L
|    | -- D

the current linking strategy (omitting the node_modules level for brevity) yields:

repo
| -- A
|    | -- D
|    | -- L′ --> symlink to ../L
| -- L
|    | -- D

So that a reference to D from within L from within A via L′ refers to D under the L at the lerna package level.

The alternative strategy I am proposing (--no-symlink?) would instead give:

repo
| -- A
|    | -- D
|    | -- L1 (copied here)
| -- L
|    | -- D

The result would be that a reference to D from within L (L1) under A would use the common D, avoiding the problems described above.

In some cases this problem might be able to be resolved with the judicious use of hoisting, such as in the case of rxjs. However, in the case of Angular, there are apparently places where it doesn’t fully respect node resolution semantics, and insists that certain packages be present locally.

Without hoisting, this approach would result in additional copies of L in each sub-repo it. Of course, that’s no different from current behavior if L were an external dependency. Perhaps more importantly, if we brought a copy of L under A as L1, then any changes to L would not be reflected in builds of A until I did another lerna bootstrap. But this is also the same as the case where L is an external dependency.

Even doing lerna bootstrap, however, would work here only if lerna copied the library/sub-repo afresh on every bootstrap. To avoid needing to do this, when copying internal dependencies according to the new strategy proposed here, lerna could consult the version number of L/package.json, compare it to the version of L already installed under A, and re-copy only if necessary. In other words, listing of internal dependencies would sort of be assumed to be of the form "L": "@latest". The programmer could then “request” that L be updated in all the sub-repos that list it as a dependency at the next lerna bootstrap by bumping the version in its package.json (or would it makes sense to have some kind of “internal” version of lerna publish for this purpose?).

However, the problem of having multiple copied L’s under all the sub-repos that list it as a dependency could be solved easily by merely listing L as hoistable. Then there would be one L (also copied) under the top-level node_modules. This will still, though, require re-lerna bootstrap-ping when changes were made to L, to ensure they are reflected in the copy at the top level.

I supposed it’s possible to view all this as a lerna monorepo behaving like its own miniature, self-contained registry. Sub-repos would be “brought in” from this “registry” in the same way as they would be brought in from an external registry. If this interpretation makes sense, then maybe the right name for the linking strategy option I am proposing should be --mock-registry.

Please let me know if I’m missing something basic, or if this issue has already been addressed in some other way.

[rtm]

Sorry about the wall of text. You are never providing enough information, except when you are providing too much. But there were some pretty pictures which I thought were relatively clear…

Bottom line: some systems don’t play nice with linking. Linking does not recreate the same semantics with regard to the handling of sub-dependencies as if the module had been brought in externally. My proposal is to provide an option where resolution behaves exactly as if the package had been brought in externally.

I will look at your lerna3 + npm5 + relative file: specifies suggestion.

BTW It’s not Angular doing module resolution tricks. It’s TypeScript.

Bob

On Thu, Jun 14, 2018 at 6:58 AM Daniel Stockman [e-mail] wrote:

i don’t understand angular’s persistence with stupid module resolution tricks.

what exactly are you trying replace the current linking with? huge wall of text doesn’t parse very well.

(lerna3 & npm5+ relative file: specifiers https://github.com/npm/npm/blob/latest/doc/spec/file-specifiers.md work great for me.)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/lerna/lerna/issues/1451#issuecomment-397137937, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfR43Nxh87jfdzaU8Kft87D9gh46gYks5t8bwigaJpZM4Ub6TW .

[rtm]

@amir-arad Unfortunately I just completed a laborious migration from yarn back to npm/lerna, because yarn was slow and unstable at least on my WSL machine. With regard to hoisting, as I mentioned

However, in the case of Angular, there are apparently places where it doesn’t fully respect node resolution semantics, and insists that certain packages be present locally.