Brunch writes output file before all source files are processed; possible race condition?

Description

We recently upgraded to Brunch 2.8.2 from the 1.x branch. Almost everything worked seamlessly, but we have one intermittent but very troubling problem.

Some JS files (in this case, files containing Angular components, but I don’t believe that’s relevant) are being omitted from our compiled build output when building in “production” mode.

Frustratingly, the problem only happens in very specific situations. We have only been able to reproduce the issue when:

• building in production mode
• on Linux
• building with a non-empty target directory (i.e. with any previous builds in the target directory; removing the entire output directory seems to work OK… but removing the individual bad file (the app.js bundle in our case) still produces an incorrect result)

Expected behavior

All files would consistently be represented in the built, minified output

Actual behavior

Some files are missing from the compiled app.js bundle. Which files seems somewhat arbitrary, but is consistent as long as we don’t change the source.

I have only these small clues:

• the missing files seem to be files deeper in the filesystem hierarchy
• no files are missing when we remove our entire output directory and re-build. If we leave old built files there, the problem manifests
• the brunch build output looks strange. It varies from run to run, even with the same source files. See below for an example of a strange-looking run. The output is not always the same, it seems to print the “compiled” lines at different points a different number of times each run. But why does it print that it’s done compiling multiple different times? Why is it using cached files when this is a build, not a watch?

$ ./node_modules/brunch/bin/brunch b --production
19 Aug 21:17:11 - info: compiling
19 Aug 21:17:16 - info: compiling.
19 Aug 21:17:17 - info: compiled 42 files into 2 files in 11.1 sec
19 Aug 21:17:25 - info: compiling
19 Aug 21:17:26 - info: compiled 203 files and 19 cached into app.js, copied 103 in 10 sec
19 Aug 21:17:34 - info: compiled 169 files and 223 cached into app.js, copied 215 in 8.1 sec
19 Aug 21:17:35 - info: compiled 13 files and 393 cached into app.js in 1.6 sec

I wish I could create a smaller repro, but the problem seems to only occur when we compile the entirety of our application which has ~500 source files in it; smaller ones don’t seem to have any problem (or running on OS X, or building in non-production mode).

However, I have managed to trim down our brunch config and package.json to try to narrow down the culprit (see below). There are very few plugins left that seem relevant, and they’re only “official” brunch plugins with default settings… which is why I believe this to be a brunch issue rather than a plugin issue, but I could be wrong about that.

Here’s a vague theory: could there be a race condition somewhere in brunch’s plugin pipeline? Basically while processing this large, deep hierarchy of source files at some point does it prematurely think it’s finished and start the next phase of the pipeline (e.g. JS minification / optimization) even though it isn’t actually done processing all source files yet? This might explain why some random files deep in the hierarchy are getting omitted, and also maybe why the build output prints that its done compiling app.js multiple times.

Environment

1. Brunch: 2.8.2
2. Node: 4.5.0
3. NPM: 3.10.6
4. Operating system: Amazon Linux (CentOS-based) Linux ((hostname)) 4.4.15-25.57.amzn1.x86_64 #1 SMP Wed Jul 27 22:37:49 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

package.json contents

{
  "name": "<our app>",
  "version": "0.0.1",
  "description": "",
  "author": "<our company>",
  "private": true,
  "devDependencies": {
    "babel-brunch": "6.0.4",
    "bower": "1.7.9",
    "brunch": "2.8.2",
    "javascript-brunch": "2.0.0",
    "css-brunch": "2.6.1",
    "sass-brunch": "2.6.3",
    "clean-css-brunch": "2.0.0"
  }
}

brunch config contents

module.exports = config:
  conventions:
    assets: [
      /^client\/assets/,
      /echidna\/shared\/assets/
    ]
  paths:
    public: '_publish'
    watched: ['client', '../echidna']
  files:
    javascripts:
      joinTo:
        'scripts/bowerfiles.js': /^bower_components/
        'scripts/app.js': [/^client/, /echidna\/shared/]
        'scripts/airbrake.js': /vendor\/airbrake/
        'scripts/modernizr.js': /vendor\/modernizr/
      order:
        before: [
          # jquery needs to come before everything else to make sure that Angular will use jQuery instead of jqLite
          'bower_components/jquery/dist/jquery.js',
          # and angular needs to come before everything but jquery to make sure it's defined when misc directives load
          'bower_components/angular/angular.js',
          # moment-timezone needs to come after moment
          'bower_components/moment/moment.js',
          'bower_components/moment-timezone/builds/moment-timezone-with-data-2010-2020.js',
          'bower_components/quill/dist/quill.js',
          # and one of our specific globals needs to come before any other code
          /echidna\/shared\/js\/InjectedCtrl/
        ]
    stylesheets:
      joinTo:
        'styles/app.css': [/^client/, /echidna\/shared/]
        'styles/bowerfiles.css': /^bower_components/
  modules:
    wrapper: false
    definition: false
  npm:
    enabled: false

Other useful files, when present (log, bower.json etc.)

I can attach our bower.json if it would help, but there is no problem with the compiled bower JS files, only our app files.

This issue did not appear prior to our brunch upgrade; we were using brunch 1.8.5 on node 0.12 with npm 2.x

Lastly, I’ve attached the output from a brunch b --production -d run. It’s quite long, but it’s the output from one of the “bad” builds.

out.txt

If there’s anything at all I can do to help debug this please let me know. I’ve spent an entire day trying to figure this out, and at this point will likely need to downgrade us back to brunch 1.8.5 so we can continue deploying successfully. I’m a big fan of brunch and would like to be on the latest and greatest, so happy to help in any way I can. Thanks!

[paulmillr]

Nevermind — this one should fix it.

[paulmillr]

@wiltzius what happens if you set config.fileListInterval to 100, 200, 300?