Cypress 10.x.x hangs under Linux + Docker using cypress/included:10.2.0

Current behavior

Hello all,

we have the problem that Cypress local runs without problems. All tests are executed and the test run includes all test specs. If we run the same code in a pipeline with the Docker image and Pipeline integration , it doesn’t work anymore. The problem is that Cypress doesn’t do anything anymore except logging the CPU usage even though there are still test specs and tests to run. Sometimes this happens during the first test spec, sometimes after another one. It is always different. Via DEBUG=cypress:* npx cypress run I logged everything and there is no error in the logs when Cypress stops working and only logs CPU usage until the pipeline times out. This looks like this in the logs:

cypress:server:util:process_profiler current & mean memory and CPU usage by process group:
  cypress:server:util:process_profiler ┌─────────┬───────────────────┬──────────────┬────────────────────┬────────────┬────────────────┬──────────┬──────────────┬─────────────┐
  cypress:server:util:process_profiler │ (index) │       group       │ processCount │        pids        │ cpuPercent │ meanCpuPercent │ memRssMb │ meanMemRssMb │ maxMemRssMb │
  cypress:server:util:process_profiler ├─────────┼───────────────────┼──────────────┼────────────────────┼────────────┼────────────────┼──────────┼──────────────┼─────────────┤
  cypress:server:util:process_profiler │    0    │    'Electron'     │      1       │       '256'        │   19.08    │     15.89      │  373.65  │    278.85    │   373.65    │
  cypress:server:util:process_profiler │    1    │ 'electron-shared' │      4       │ '36, 192, 37, 213' │   16.49    │     13.73      │  254.15  │    245.68    │   260.08    │
  cypress:server:util:process_profiler │    2    │     'cypress'     │      1       │        '30'        │   55.27    │     47.89      │  248.22  │    243.28    │   255.86    │
  cypress:server:util:process_profiler │    3    │     'plugin'      │      1       │       '238'        │    0.05    │      2.53      │  146.84  │    171.95    │    241.1    │
  cypress:server:util:process_profiler │    4    │     'ffmpeg'      │      1       │       '251'        │    1.25    │      1.2       │  76.43   │    72.46     │    76.43    │
  cypress:server:util:process_profiler │    5    │      'other'      │      2       │    '999, 1000'     │     0      │      0.03      │   3.4    │     3.39     │    3.45     │
  cypress:server:util:process_profiler │    6    │      'TOTAL'      │      10      │        '-'         │   92.14    │     80.68      │  1102.7  │    999.74    │   1106.26   │
  cypress:server:util:process_profiler └─────────┴───────────────────┴──────────────┴────────────────────┴────────────┴────────────────┴──────────┴──────────────┴─────────────┘ +10s

I can’t tell for sure since the logs don’t indicate anything but it seems like the test runner is crashing or Cypress is losing connection to it, or it is a memory leak, but unfortunately it is impossible to tell from the missing log entries. It is very hard to trace but happens with every test run. We have had this problem for over a year now…

I can’t tell for sure since the logs don’t indicate anything but it seems like the test runner is crashing, the Chrome crahed (Aw, snap!) or Cypress is losing connection to it. It is very hard to trace but happens with every test run. We have had this problem for over a year now…

If you run the Docker image locally, without pipeline, then it works as well. So it must be the interaction between pipeline and Docker image.

Currently we are using Cypress 10.2.0 with Chrome 100, but we had the problem also with Cypress 8.3.0 This happens also in Electron Browser. i’ve tried "video": false, "numTestsKeptInMemory": 1 and 0, and our index.js in cypress/e2e/plugins looks like this:

module.exports = (on, config) => {
  on("before:browser:launch", (browser = {}, launchOptions) => {
    if (browser.name === "chrome") {
      launchOptions.args.push("--disable-gpu");
      launchOptions.args = launchOptions.args.filter(
        (Element) => Element !== "--disable-dev-shm-usage"
      );
    }
    return launchOptions;
  });
};

Desired behavior

Cypress no longer crashes during the pipeline run or spits out proper logs, after which you can see what the problem is.

Test code to reproduce

Since it fails on a different spec each time, it doesn’t seem to be related to actual test code

Cypress Version

10.x.x

Other

Cypress seems to be very resource-heavy. Our local computers run with 8 CPUs and the pipeline with 6 CPUs. in my opinion it cannot be due to our hardware resources

[PeterDekkers]

On top of disabling the Command Log, disabling video recording has helped me to greatly reduce the frequency of hanging tests.

[nagash77]

possibly related to https://github.com/cypress-io/cypress/issues/23391 which is being actively worked on. We will double check to see if this issue is resolved when work on the linked ticket is complete.