Ava slows when there are many non-run files

Description

I started a project a while ago and I noticed that ava slowed down over time. At the start it used to clock in at under a second. As I added more code and my configuration changed it is now around 15 seconds. This is an extremely long feedback cycle. Using --watch does not appear to produce a significant improvement.

I managed to reproduce the problem in a test repo. Further testing show it has to do with the number of files in the directory. It does not matter what type of files they are. In the example repo, I have a 1 simple test, a copy of node, and a benchmark script. When I remove the copy of node it is significantly faster. Here is a sample run of the script.

user@computer:~/src/ava-slow-test% ./benchmark.sh
Running tests...

  1 passed


real	0m7.377s
user	0m6.863s
sys	0m1.379s
Removing lib and re-running tests...

  1 passed


real	0m0.737s
user	0m0.589s
sys	0m0.100s
Resetting lib...

Environment

user@computer:~/src/ava-slow-test% node -e "var os=require('os');console.log('Node.js ' + process.version + '\n' + os.platform() + ' ' + os.release())"
Node.js v7.4.0
darwin 16.4.0
user@computer:~/src/ava-slow-test% ./node_modules/.bin/ava --version
0.18.2
user@computer:~/src/ava-slow-test% npm --version
4.0.5

Links

Test Repository

[novemberborn]

Globbing files reported 13.409ms and globbing helpers reported 6406.377ms.

Wow, that’s unexpectedly high.

I looked around and I didn’t see a way to change the helpers patterns. Assuming I didn’t miss it, would this be a feature worth adding?

Actually I think we need to optimize how we find helpers. The search starts here: https://github.com/avajs/ava/blob/bd5ed603356f7a038fed62f64494b35598741d67/lib/ava-files.js#L138, which is called from https://github.com/avajs/ava/blob/bd5ed603356f7a038fed62f64494b35598741d67/api.js#L123 and in turn https://github.com/avajs/ava/blob/bd5ed603356f7a038fed62f64494b35598741d67/api.js#L149.

It strikes me that we already know where the test files are when we start looking for helpers. We can find test and __test__ directories based on that list, and then search for helpers/**/*.js and **/_*.js files inside those directories. That should prevent us from searching your entire project tree.

I’m hesitant when it comes to making helper patterns configurable. I think we first need to get the default behavior right, lest the answer becomes “nah just change these patterns”.

For what it is worth, if there were some debugs in the code with the timings for the globs and I could out find how to configure the default helper patterns, it may have prevented me from reporting this issue.

I think it’s a bug actually, so I’m glad you reported it! 😄

[novemberborn]

I think we should add some options to filter the paths before passing it to the matcher. Like --exclude -e

That’s what ! is supposed to do, but it looks like globby is still spending too much time searching. Perhaps related to https://github.com/sindresorhus/globby/issues/43.