pnpm install should be easy and documented for Vaadin projects

After I read in the Vaadin docs that Vaadin uses pnpm as the client-side package manager (instead of npm), I want to add dependencies with pnpm install. That allows me to use any examples from the pnpm documentation in my Vaadin projects “as-is”. At the moment that does not work because Vaadin requires a particular version of pnpm (4.5.0) with a particular configuration (--shamefully-hoist=true). Any attempt to use a different pnpm version or a different configuration leads to an incompatible node_modules folder structure (i.e. the app start / production build fails).

Steps to reproduce:

1. create a new Vaadin 18 / Fusion project with https://start.vaadin.com/?preset=fusion (or with Vaadin CLI: npx @vaadin/cli init --fusion pnpm-install-test)
2. trigger generation of the ‘special’ Vaadin npm packages (like @vaadin/flow-frontend and @vaadin/form): mvn vaadin:prepare-frontend
3. install a dependency with the latest pnpm (5.13.7): npx pnpm install mobx
4. run mvn to start the app in the dev mode
5. see a run-time error:

   2020-12-23 15:24:40.679  INFO 79222 --- [         task-1] c.v.flow.server.frontend.FrontendTools   : using '/Users/viktor/.vaadin/node/node /Users/viktor/.vaadin/node_modules/pnpm/bin/pnpm.js' for frontend package installation
   2020-12-23 15:24:41.705 ERROR 79222 --- [         task-1] c.v.f.s.frontend.TaskUpdatePackages      : Command `/Users/viktor/.vaadin/node/node /Users/viktor/.vaadin/node_modules/pnpm/bin/pnpm.js --shamefully-hoist=true install` failed:
   Using hooks from: /Users/viktor/Downloads/pnpm-install-test/pnpmfile.jsreadPackage hook is declared. Manifests of dependencies might get overridden ERROR  This "node_modules" folder was created without the --shamefully-hoist option. You must remove that option, or else "pnpm install --force" to recreate the "node_modules" folder.
   2020-12-23 15:24:41.705 ERROR 79222 --- [         task-1] c.v.f.s.frontend.TaskUpdatePackages      : >>> Dependency ERROR. Check that all required dependencies are deployed in pnpm repositories.
   

6. delete node_modules and install a dependency with pnpm 4.5.0, but without --shamefully-hoist=true

   rm -rf node_modules
   npx -p pnpm@4.5.0 pnpm install mobx
   

7. See the same error

Expected behavior:

• pnpm i mobx --shamefully-hoist / npx pnpm i mobx --shamefully-hoist works in the scenario described above with the latest version of pnpm.
• building a project after a Vaadin version upgrade that changes some of the pinned platform npm dependency versions works without issues (see #8434)
• removing the --shamefully-hoist option is a separate task: https://github.com/vaadin/flow/issues/9834

Workaround: Use npx -p pnpm@4.5.0 pnpm --shamefully-hoist=true install [dependency] to add an npm dependency.

[platosha]

These cross-app dependency sharing questions sound really tricky to resolve. Could we stop using our shared pnpm installation in ~/.vaadin altogether, so that we don’t have this sort of questions anymore?

From npx description:

If any requested packages are not present in the local project dependencies, then they are installed to a folder in the npm cache, which is added to the PATH environment variable in the executed process. A prompt is printed (which can be suppressed by providing either --yes or --no).

Sounds like if matching pnpm is missing globally, Vaadin could use the pnpm version it needs through something like this prefix:

$ npx --yes --package=pnpm@${requiredVersion} -- pnpm install

The command above runs pnpm install with the specified pnpm version. Several benefits here:

• npx does all the download and “installation” hassle automatically. npx itself does not need to be installed, as it is bundled with npm, which is bundled with node.
• pnpm only goes to npm cache, which is neither node_modules in the project nor global node_modules. The ”installation” does not affect actually installed node_modules anywhere.
• npm cache is already versioned, no version conflicts to face from sharing a single directory. We don’t have to invent versioning in ~/.vaadin either.
• npm cache is already shared. If a developer has several LTS apps, requiring pinned pnpm 4.5.0, pnpm will be still “installed” only once.

[platosha]

One solution could be to up the minimum version of V19 to one with compatible node_modules (5 or newer?) in V19.

Sounds like the way to go, IMO.