[Documentation]: CLI --filter option might no be clear enough / Filter {filterPath} did not return a valid test list

Version

27.4.5

Steps to reproduce

Steps : (1) Go to the documentation about filter file (2) Create a working project with a valid jest config (3) Create a filter file as stated in the doc to be used with bash --filter=<filename> (4) Try a few functions (5) Receive “Filter {filterPath} did not return a valid test list”

Expected behavior

I expected the documentation to provide what I was supposed to write in the file, that is :

“[a] Path to a module exporting a filtering function.”

One could expect a function of the type

function filteringFunction(files:string[]):string[]{
  return files.filter( /** logic **/ ) 
}

Or even :

function isFileValid(file:string):boolean {
  return applyLogic(file)
}

Actual behavior

But, because the code is the following :

if (filter) { 
      const tests = searchResult.tests;
      const filterResult = await filter(tests.map(test => test.path));

      console.log({ filterResult })

      if (!Array.isArray(filterResult.filtered)) {
        console.log(filterResult.filtered)
        throw new Error(
          `Filter ${filterPath} did not return a valid test list`
        );
 }

and also

 const filteredSet = new Set(
        filterResult.filtered.map(result => result.test),
      );

      return {
        ...searchResult,
        tests: tests.filter(test => filteredSet.has(test.path)),
      };

I got it to work by using the very undocumented:

module.exports = function filter(testPaths) {
  const filtered = testPaths
    .filter((testPath) => isFiltered(testPath))
    .map((testPath) => ({ test: testPath }));

  return {
    filtered,
  };
};

I also discovered it was AN ASYNC function 😃

Additional context

I would suggest first to maybe update the documentation with something of the sort:

--filter=<file>

Path to a module exporting a filtering function. This asynchronous function receives a list of paths which can be manipulated to exclude tests from running by returning an object with the “filtered” property. Especially useful when used in conjunction with a testing infrastructure to filter known broken.

Example:

// my-filter.js
module.exports = (testPaths) => {
  const allowedPaths = filterPaths(testPaths) // ["path1.spec.js", "path2.spec.js", etc]
  return {
      filtered : allowedPaths,
   }
}

And (but I don’t know if it’s possible), to edit the set creation

const filteredSet = new Set(
   ❌ filterResult.filtered.map(result => result.test) 
   ✅ filterResult.filtered
);

Environment

System:
    OS: macOS 12.0.1
    CPU: (8) x64 Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz
  Binaries:
    Node: 14.17.0 - ~/.nvm/versions/node/v14.17.0/bin/node
    Yarn: 1.22.15 - ~/.yarn/bin/yarn
    npm: 6.14.13 - ~/.nvm/versions/node/v14.17.0/bin/npm
  npmPackages:
    jest: ^27.4.5 => 27.4.5

[gramsco]

@SimenB Done! Waiting for the CLA.

[mrazauskas]

Just to leave a note. Jest’s end-to-end tests is a great usage reference too: https://github.com/facebook/jest/tree/main/e2e/filter