BackstopJS Report doesn't fire up after more than 999 tests

Hey @garris,

At 1000 tests and more, the BackstopJS Report doesn’t work anymore. I tested it locally at 999 it works and at 1000 it fails.

I don’t have any errors.

This what I get in the CMD :

CasperJS:  Ready event received.
CasperJS:  Current location is http://localhost:4000/documentation/3.3/composantes.html?backstop=tru
e#graphiques
CasperJS:  Capturing screenshots for desktop (1280x1014)
CasperJS:  Ready event received.
CasperJS:  Current location is http://localhost:4000/documentation/3.3/composantes.html?backstop=tru
e&touch=true#graphiques
CasperJS:  Capturing screenshots for phone (320x480)
CasperJS:  Ready event received.
CasperJS:  Current location is http://localhost:4000/documentation/3.3/composantes.html?backstop=tru
e&touch=true#graphiques
CasperJS:  Capturing screenshots for tablet_v (768x1024)
CasperJS:  Ready event received.
CasperJS:  Current location is http://localhost:4000/documentation/3.3/composantes.html?backstop=tru
e&touch=true#graphiques
CasperJS:  Capturing screenshots for tablet_h (1024x768)
CasperJS:  Ready event received.
CasperJS:  Current location is http://localhost:4000/documentation/3.3/composantes.html?backstop=tru
e&touch=true#graphiques
CasperJS:  Capturing screenshots for desktop (1280x1014)
CasperJS:  Comparison config file updated.

Bitmap file generation completed.
COMMAND | Executing core for `report`

C:\Projets\N3\fwd-webapp-overlay-skin>

Thanks!

[garris]

@stephenbe I just wanted to follow up. I attempted to throttle concurrent comparisons but the memory leak persisted. I committed this code here https://github.com/garris/BackstopJS/commit/adc2ae2373bdeb99fea6bb0f678da8c044127330 even though this approach did not immediately solve the issue. What I believe is that each resemble object is generating and caching a comparison image – holding that in RAM while all comparisons are running. I think the next thing to try is to refactor this flow such that references to each resemble object are completely cleaned up once a write to fs is complete. I am hoping that would make backstop MUCH more RAM efficient. The good news is that once map is complete all instances are cleaned up and memory is completely returned. So I am confident that this issue is addressable. Will just take a little effort to get the sequencing right.

Would appreciate any help here from anyone who is interested. Otherwise I will try to get to this in January.

[garris]

Ok. I was able to run the app. After bitmaps were generated the comparison process ran and grew steadily to 5+Gig. Still investigating.