Tests causing CI agents to run out of memory on cypress 8.2 & above

Current behavior

The Azure CI Agent machine running out of memory when running cypress tests on 8.2 & above on Azure Pipeline CI agents (Linux OS). The cypress dashboard eventually times out as well.

<— Last few GCs —>

[31:0x1c7300000000]  8599171 ms: Mark-sweep 1835.4 (2072.0) -> 1832.6 (2071.6) MB, 205.8 / 0.0 ms  (average mu = 0.263, current mu = 0.223) allocation failure scavenge might not succeed
[31:0x1c7300000000]  8599492 ms: Mark-sweep 1835.0 (2072.0) -> 1832.6 (2071.6) MB, 294.6 / 0.0 ms  (average mu = 0.166, current mu = 0.082) allocation failure scavenge might not succeed

<— JS stacktrace —>

The Test Runner unexpectedly exited via a exit event with signal SIGSEGV

Please search Cypress documentation for possible solutions:

https://on.cypress.io

Check if there is a GitHub issue describing this crash:

https://github.com/cypress-io/cypress/issues

Consider opening a new issue.

----------

Platform: linux (Debian - 10.10)
Cypress Version: 8.3.0

Desired behavior

No response

Test code to reproduce

This has been consistently reproducible on 8.2 and 8.3. Running the same spec files on the same agents on version 8.1 and below works as expected.

Cypress Version

8.3

Other

No response

[DJSdev]

Hey, my company has this issue too and I think I found the cause and am close to having a PR ready to fix this.

Setup

In my environment, I found this out of memory issue only happens in Chrome, it doesn’t seem to occur in FireFox. Hopefully someone might be able to confirm this for me, as this would make me hopeful my fix solves this issue.

According to Task Manager, Cypress’s server was growing to multiple GB in size, so I added some code to make Cypress dump its server’s heap memory every 90 seconds. In the contents of that, I found a memory leak and I hope it’s the one related to this issue.

Investigation

In the memory dump, I found Cypress was storing hundreds (maybe thousands) copies of the same base64 encoded string in an array named pendingBrowserPreRequests.

I looked at my company’s app and found a css file that has the base64 font file in question hardcoded as the src field.

Side note: this issue could be remedied on the application side if base64 font files are removed from css files and changed to urls.

Root Cause

Cypress subscribes to the CDP event Network.requestWillBeSent and adds network requests that will be sent to an array. The elements in the array are removed when the corresponding response for the request comes back.

With these fonts inside the URL field of css stylesheets, this triggers Chrome to send a Network.requestWillBeSent event to Cypress with the URL of the request being the entire base64 encoded font file (lol). These requests are not actually going to be sent from the browser as it will parse these values instead, so Cypress never knows when to release these from its array since it never sees a response. It could be argued that this is a Chrome issue, but the fix should be relatively simple on the Cypress side.

I think I can have a PR out tomorrow to fix.

cc: @jennifer-shehane @brian-mann

[DJSdev]

@BlueWinds PR is out!