BackstopJS test execution failing: "Error: Stream not writable"

So I’ve just started tinkering with BackstopJS. I setup some basic tests and was trying to work it into my normal build process in gulp.

For some reason though, the script just stopped executing correctly. The test worked fine several times earlier today, I came back later, made some CSS changes in my project, ran another test, and received this error:

Bitmap file generation completed.
COMMAND | Executing core for `report`
events.js:141
      throw er; // Unhandled 'error' event
      ^

Error: Stream not writable
    at ChunkStream.write (/Users/*****/Sites/*****/node_modules/pngjs/lib/chunkstream.js:46:24)
    at PNG.write (/Users/*****/Sites/*****/node_modules/pngjs/lib/png.js:92:16)
    at ReadStream.ondata (_stream_readable.js:525:20)
    at emitOne (events.js:77:13)
    at ReadStream.emit (events.js:169:7)
    at readableAddChunk (_stream_readable.js:146:16)
    at ReadStream.Readable.push (_stream_readable.js:110:10)
    at onread (fs.js:1738:12)
    at FSReqWrap.wrapper [as oncomplete] (fs.js:576:17)

For comparison, here is the same output from earlier today:

Bitmap file generation completed.
COMMAND | Executing core for `report`
compare | ERROR { size: isDifferent, content: 40.95%, threshold: 0.1% }: ***** *****_0_document_0_phone.png
   See: /Users/*****/Sites/*****/backstop_data/bitmaps_test/20161115-135709/failed_diff_*****_0_document_0_phone.png
compare | OK: ***** *****_0_document_0_phone.png
compare | OK: ***** *****_0_document_0_phone.png
compare | ERROR { size: ok, content: 1.06%, threshold: 0.1% }: ***** *****_0_document_1_tablet_v.png
   See: /Users/*****/Sites/*****/backstop_data/bitmaps_test/20161115-135709/failed_diff_*****_0_document_1_tablet_v.png
compare | OK: ***** *****_0_document_1_tablet_v.png
compare | OK: ***** *****_0_document_1_tablet_v.png
compare | ERROR { size: ok, content: 0.60%, threshold: 0.1% }: ***** *****_0_document_2_tablet_h.png
   See: /Users/*****/Sites/*****/backstop_data/bitmaps_test/20161115-135709/failed_diff_*****_0_document_2_tablet_h.png
compare | OK: ***** *****_0_document_2_tablet_h.png
compare | OK: ***** *****_0_document_2_tablet_h.png
 report |
          Test completed...
 report | 6 Passed
 report | 3 Failed

I tried deleting the html_report directory and rerunning the script. That didn’t work. I tried deleting bitmaps_reference and bitmaps_test. It could recreate the images, but the test failed in the same fashion.

There’s also nothing fancy about the backstop.json file. I think it’s just adapted from one of the demos:

{
  "id": "*****",
  "viewports": [
    {
      "name": "phone",
      "width": 320,
      "height": 480
    },
    {
      "name": "tablet_v",
      "width": 568,
      "height": 1024
    },
    {
      "name": "tablet_h",
      "width": 1024,
      "height": 768
    }
  ],
  "scenarios": [
    {
      "label": "*****",
      "url": "*****/*****/index.html",
      "readyEvent": "backstopjs_ready",
      "misMatchThreshold" : 0.1,
      "onBeforeScript": "onBefore.js",
      "onReadyScript": "onReady.js"
    },
    {
      "label": "*****",
      "url": "*****/*****/index.html",
      "readyEvent": "backstopjs_ready",
      "misMatchThreshold" : 0.1,
      "onBeforeScript": "onBefore.js",
      "onReadyScript": "onReady.js"
    },
    {
      "label": "*****",
      "url": "*****/*****/index.html",
      "readyEvent": "backstopjs_ready",
      "misMatchThreshold" : 0.1,
      "onBeforeScript": "onBefore.js",
      "onReadyScript": "onReady.js"
    }
  ],
  "paths": {
    "bitmaps_reference": "backstop_data/bitmaps_reference",
    "bitmaps_test": "backstop_data/bitmaps_test",
    "casper_scripts": "backstop_data/casper_scripts",
    "html_report": "backstop_data/html_report",
    "ci_report": "backstop_data/ci_report"
  },
  "casperFlags": [],
  "engine": "phantomjs",
  "report": ["browser"],
  "debug": false
}

Does anyone have any ideas?

[aczekajski]

I’m getting the same error even though sanity check does work correctly.

I tried debugging it and the last place where code executes correctly is the compare-resemble.js file, up to the line with comparison.onComplete. Callback passed to this method never executes, because of “Stream not writable” happening inside of node-resemble-js.

So, this seems to be a bug in node-resemble-js or even deeper. One thing that might be done on the backstopjs side is to catch such error and handle it somehow. Current behavior is very annoying, because backstop hangs forever during the “report” phase.

Unfortunately, node-resemble-js has no onError method so the only place where backstopjs can recognize that something went wrong, is in core/util/compare/index.js file, in compareImages method. The following code properly rejects a promise if the child comparison process ended with error (ie. because of uncaught “Stream not writable”).

worker.on('exit', function (code, signal) {
  if (code > 0) {
    logger.error('EXECUTION ERROR');
    reject('There was an error in child process');
  }
});

However, I didn’t check what impact this rejection will have on the rest of the code. Good thing is that it doesn’t hang with this lines added.

[benbcai]

Thank you @mirzazeyrek for publishing a new release!