[Bug]: Unable to run a test from whitelisted "node_modules"

Version

27.0.6

Steps to reproduce

1. Clone the repo at https://github.com/kettanaito/jest-nm-issue
2. yarn install
3. yarn test

Expected behavior

Jest detects the node_modules/__test/example.test.js file and runs it. The file matches the testMatch pattern, which you can confirm by clicking on that pattern in the terminal and being navigated to the correct existing directory (tested in VS Code).

Actual behavior

$ jest
No tests found, exiting with code 1
Run with `--passWithNoTests` to exit with code 0
In /Users/kettanaito/Projects/contrib/jest-nm-issue
  2 files checked.
  testMatch: /Users/kettanaito/Projects/contrib/jest-nm-issue/node_modules/__test/*.test.js - 0 matches
  testPathIgnorePatterns: not_node_modules - 2 matches
  testRegex:  - 0 matches
Pattern:  - 0 matches

• 0 matches next to the correct pattern pointing to an existing directory under node_modules is not expected.

Additional context

I distribute a test suite as a node module because it acts as a specification for multiple projects. I wish to run that test suite from the node_modules where it’s installed.

Regardless of my setup, it’s confusing that Jest seems to ignore anything as long as it includes “node_modules” even if I’ve explicitly whitelisted a certain directory in it.

Environment

System:
  OS: macOS 11.5.1
  CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
Binaries:
  Node: 12.18.2 - /usr/local/bin/node
  Yarn: 1.22.10 - /usr/local/bin/yarn
  npm: 6.14.5 - /usr/local/bin/npm
npmPackages:
  jest: ^27.0.6 => 27.0.6

[tomalec]

I’d like to add another use case here.

We are running a platform (WordPress > WooCommerce) with an ecosystem of plugins/extensions. As platform developers, we created a package containing a suite of tests (many tests in many files).

Now, we would like plugin developers (within our company and from the community) to consume this package. Run the suite, to make sure their plugin integrates well with the platform and serves a consistent UX.

The problem is how one can consume test suites from the npm package?:

1. Add the node module to the allow-list

   • Not possible as per this issue.

2. Import them and call from a local file:

   import { runAllSuites } from `@woocommerce/admin-e2e-tests`;
   runAllSuites();
   

   • Fails as well. The above will join them to a single suite, then the report will be shown only once the entire file=> all the suites => all the tests are run (see https://github.com/facebook/jest/issues/6616#issuecomment-669078979. The consumers would have to wait many minutes to notice what’s happening. Being stuck with:

   > npm run test
   RUNS  ../../../tests/specs/imported.test.js
   
   Test Suites: 0 passed, 2 total
   Tests:       40 passed, 385 total
   Snapshots:   0 total
   Time:        235.221s
   

   Plus, most probably will hit the timeout too early.

The only fallback I can think of is to ask consumers, to clone the test file structure, and import every single suite separately. What’s extremely fragile for the maintenance updates, requires manual labor, and pollutes the consumer’s files structure, impairing DevX.

[ashleykolodziej]

I’ll chime in with my use case - I’m a professor using GitHub classroom to automatically provision repositories for my students to work on, and I use Jest to help me with grading to ensure students have met specific requirements in their code. I keep my tests in a separate repository, and make it a dependency in node_modules, so that if a student discovers a bug in my test after the class starts the assignment, I can update it once for the class and not have to update 20+ individual repositories.

Here’s an example of how I have my tests listed as a dependency: https://github.com/ProfessorKolodziej/cm523-hello-world/blob/main/package.json#L79

How I configured Jest to run tests from node modules in a hacky way that I know is going to fail next time I update Jest because providesModuleNodeModules is removed in version 26: https://github.com/ProfessorKolodziej/cm523-hello-world/blob/main/package.json#L85-L112

And then the actual test for this assignment that I run across all my class’s repositories is here: https://github.com/ProfessorKolodziej/cm523-unit-tests/blob/main/hello-world/index.test.js

Right now, this setup works fine for anyone on a Mac or Ubuntu, but I just had a student report that this is broken on Windows, and I’m not sure how to keep my couple of students on Windows moving now. They run a test command locally before pushing changes to GitHub to more quickly check that their requirements are in place.

Being able to pull a test from a specific repository in node_modules in a more officially supported way would be an ENORMOUS help to me.