[Bug] Cannot add dependency as tar.gz from URL.

• I’d be willing to implement a fix

Describe the bug

GitPKG is a website which allows the installation of NPM packages from a github repo which is a monorepo, i.e. the individual packages to be installed are located in subdirectories of the main repo. This is a situation that typically cannot be resolved by package managers, since the URL of the repo itself cannot be distinguished from a subdirectory of that repo.

In Yarn v1, the following line can be run:

yarn add https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello

This will add the hello repo to the project’s dependencies. Accessing the above URL directly will return a tar.gz file containing the repo data.

In Yarn v2, this command will fail with the following stack trace:

> yarn add https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello
Internal Error: Invalid descriptor (https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello)
    at Module.x (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:2:434115)
    at D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:2:29878
    at Array.map (<anonymous>)
    at M.execute (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:2:29786)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async M.validateAndExecute (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:2:659490)
    at async Y.run (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:17:3854)
    at async Y.runExit (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:17:4021)
    at async h (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:2:284223)
    at async r (D:\eric\Documents\Programming\Website\test-icons\.yarn\releases\yarn-berry.cjs:2:282834)

I researched the issue here and found https://github.com/yarnpkg/berry/issues/994#issuecomment-628586986, but running yarn add hello@https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello provides the same error.

To Reproduce

I have reproduced the error, using Sherlock on my local machine.

Reproduction

This is my reproduction case:

// The following command was valid 
// yarn add https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello

const installPromise = yarn(`add`, `https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello`);

// Should install successfully but can't.
await expect(installPromise)
  .resolves.toBeTruthy();

/**
 * Specifying the package name doesn't work either.
 * yarn add hello@https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello
 * @see: https://github.com/yarnpkg/berry/issues/994#issuecomment-628586986
 */

const installPromise = yarn(`add`, `hello@https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello`);

await expect(installPromise)
  .resolves.toBeTruthy();

Environment if relevant (please complete the following information):

• OS: Windows
• Node version: v14.15.4
• Yarn version: 2.4.0

Additional context

I was not able to get Sherlock working properly on the sandbox website due to the following error:

Usage Error: This tool requires a Node version compatible with >=12 <14 || 14.2 - 14.9 || >14.10.0 (got 10.23.0). Upgrade Node, or set `YARN_IGNORE_NODE=1` in your environment.·
Yarn Package Manager - 2.4.0.dev·

I was able to clone the project to execute Sherlock offline and create the above reproduction.

[arcanis]

This is expected; we currently require the package name to be listed, except for local file paths. ie, this isn’t supported:

yarn add https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello

This is:

yarn add hello@https://gitpkg.now.sh/EqualMa/gitpkg-hello/packages/hello

I’ll clarify the error message in the next release, I agree it’s currently quite confusing.

[refi64]

Just as a quick FYI, if anyone here has a site where adding the .tgz is not an option, it seems you can just add it after a #, e.g.:

yarn add https://the-url.com/package#.tgz

in which case it usually isn’t counted as part of the URL, so the actual URL is https://the-url.com/package, but Yarn sees the .tgz. (This trick is often used to rename sources in RPM spec files.)