[Bug] Some bin scripts not working in CI environment with `nodeLinker: node-modules`

Thank you for your work on yarn and on-going improvements. Caveat, I am just upgrading to yarn 2 so this may be down to some naïvety on my part. Thank you in advance for your help.

• I’d be willing to implement a fix

Describe the bug

I am doing a migration from yarn1 to yarn2 for my workspaced monorepo. I have upgraded the version to yarn-berry and still only using nodeLinked: node-modules. The problem I’m having is that some of my existing scripts that execute packages from the bin directory do not work when they are in a CI environment, however they do work locally.

Example script

"lint": "eslint --ext js,ts,tsx src --max-warnings=0"

Locally this executes as expected, on CI the output is: command not found: eslint. Note, this has also happened with a script that uses ts-node, so is not exclusive to eslint.

Workaround

I have verified that the eslint package is actually being installed on the CI environment and it seems I can get around this by invoking the bin directly, as such:

"lint": "../../../node_modules/.bin/eslint --ext js,ts,tsx src"

This is a sub-optimal solution for a few reasons but proves the lint package is there in bin and able to be executed. It feels like yarn is unable to resolve to the bin directory for some reason, maybe because of workspaces?

Only some packages

As I mentioned, this doesn’t seem to happen for everything, we also have a prettier script that runs which is working fine:

"pretty-check": "prettier --config ./.prettierrc --list-different './**/*.{js,json,ts,tsx,md,yml}'"

My understanding is that bin scripts like this need to be treated differently PnP linker is being used, but I would think there is no impact if still using node-modules?

To Reproduce

Minimum reproduction repo available here: https://github.com/bhishp/yarn-2-workflow-bin-repro

I fresh-installed yarn 2 with a workspace config and two create-react-app packages and implemented some of the scripts mentioned above. Locally all the scripts work but once in CI environment the problems occur.

Failing build with lint scripts: https://github.com/bhishp/yarn-2-workflow-bin-repro/runs/1776495972?check_suite_focus=true

Failing build with ts-node script (though explicitly added as a dependency): https://github.com/bhishp/yarn-2-workflow-bin-repro/runs/1776516171?check_suite_focus=true

Failing build with prettier script (though explicitly added as a dependency): https://github.com/bhishp/yarn-2-workflow-bin-repro/runs/1776471730

☝🏽 This prettier script is working in my actual project but failing in this reproduction

Screenshots

Workflow output of failed script, command not found: eslint:

Environment if relevant (please complete the following information):

• OS: GitHub workflows runs-on: ubuntu-latest
• Node version: 14.15.1
• Yarn version: 2.4.0

Additional context

This monorepo consists of 4 packages, 3 of which are CreateReactApp projects, mentioned because CRA bundles eslint (though this issue happens with other bins too).

yarnrc.yml for reference:

nodeLinker: node-modules

npmRegistries:
  //npm.pkg.github.com:
    npmAlwaysAuth: false

npmRegistryServer: "https://registry.yarnpkg.com"

npmScopes:
  org:
    npmPublishRegistry: "https://npm.pkg.github.com"
    npmRegistryServer: "https://npm.pkg.github.com"

plugins:
  - path: .yarn/plugins/@yarnpkg/plugin-workspace-tools.cjs
    spec: "@yarnpkg/plugin-workspace-tools"

yarnPath: ".yarn/releases/yarn-berry.cjs"

root-level package.json:

{
  "private": true,
  "workspaces": [
    "packages/ui",
    "packages/*/ui"
  ],
  "resolutions": {
    "webpack": "4.44.2"
  },
  "devDependencies": {
    "cypress": "6.2.1"
  }
}

[andreialecu]

Additionally, the build shows that ts-node and prettier are both failing, even though they are explicitly listed as dependencies.

They’re listed as dependencies in your root, but they should be dependencies in each workspace as well.

Alternatively you can define the script in the root package.json and use the $INIT_CWD env variable to make it work on your target workspace. See: https://yarnpkg.com/getting-started/qa#how-to-share-scripts-between-workspaces

[bhishp]

Thank you both.

TLDR Resolution for reference.

In a workspaced environment - even if using nodeLinker: node-modules - make sure to list any and all dependencies that your scripts rely on for each workspace. Alternatively, create shared scripts (say in the root package.json), where these dependencies are listed and re-use them from within your workspaces, as described here: https://yarnpkg.com/getting-started/qa#how-to-share-scripts-between-workspaces

Rest…

@andreialecu adding the dependency to the specific workspace fixed the issue, as did using a shared script - an example of shared script included below for reference:

In root package.json

"scripts": {
    "g:lint": "cd $INIT_CWD && eslint --ext js,ts,tsx src",
    "g:pretty-check": "cd $INIT_CWD && prettier --list-different './**/*.{js,json,ts,tsx,md,yml}'"
  },
"devDependencies": {
  "eslint": "^7.18.0",
  "prettier": "^2.2.1",
  "ts-node": "^9.1.1"
}

In pkg1 package.json

  "scripts": {
    "lint": "yarn g:lint",
    "pretty-check": "yarn g:pretty-check"
  },

(green build: https://github.com/bhishp/yarn-2-workflow-bin-repro/runs/1777056188)

@arcanis thank you for the explanation, is it worth adding something to the migration guide for this? Anything I can help with?

Closing this issue, thanks for your help