Tests that iterate over many DOM elements with `.each()` run slower in Electron during `cypress run` in 3.8.0

Current behavior:

We have a storybook cypress smoke test that opens all the stories and checks that they render. The test itself is very simple:

it('can load all components', () => {
    cy.get('[id^=explorer]').each($component => {
      cy.wrap($component)
        .click()
        .get(`[id^=${$component[0].id}--]`)
        .each($story => {
          cy.wrap($story)
            .click()
            .wait(50)
            .componentHasLoaded()
        })
    })
  })

With

Cypress.Commands.add('componentHasLoaded', () => {
  cy.get('#storybook-preview-iframe').then($iframe => {
    const doc = $iframe.contents()
    cy.wrap(doc.find('#root')).should('not.be.empty')
  })
})

Our test duration went from roughly 2 minutes to roughly 8 minutes as seen in the graph: (after some commits we decided to roll back to cypress@3.7.0)

There were absolutely no changes in the app code.

The only change was moving from cypress@3.7.0 to cypress@3.8.0. From what we’ve seen, there are no noticeable changes in the test (no new logs or errors) other than the interaction just being slower. The command to run cypress is exactly the same but we’ve noticed that the Electron version has changed from 73 to 78 as seen in the screenshots:

Desired behavior:

We’d like to know if there is something to do on our end to go back to the old speed or if there is a bug in cypress that created this performance regression.

Steps to reproduce: (app code and test code)

Run a storybook test with:

it('can load all components', () => {
    cy.get('[id^=explorer]').each($component => {
      cy.wrap($component)
        .click()
        .get(`[id^=${$component[0].id}--]`)
        .each($story => {
          cy.wrap($story)
            .click()
            .wait(50)
            .componentHasLoaded()
        })
    })
  })

With

Cypress.Commands.add('componentHasLoaded', () => {
  cy.get('#storybook-preview-iframe').then($iframe => {
    const doc = $iframe.contents()
    cy.wrap(doc.find('#root')).should('not.be.empty')
  })
})

Versions

cypress@3.8.0

Performance regression found in

Build-agent version 1.0.22569-ba1c5aa5 (2019-12-16T14:39:46+0000)
Docker Engine Version: 18.09.6
Kernel Version: Linux b9cc89c6eb3c 4.15.0-1052-aws #54-Ubuntu SMP Tue Oct 1 15:43:26 UTC 2019 x86_64 Linux
Starting container cypress/base:12.6.0
  image cache not found on this host, downloading cypress/base:12.6.0

Also locally in MacOS 10.15.2

[fr0]

I used the Chrome dev tools Performance tab to profile one of my tests running under 3.7.0 vs 3.8.3. There is a definite difference in the code path.

3.7.0:

3.8.3

It seems that 3.8.0 might’ve introduced some change in the isVisible check? The root cause of the extra slowness in this particular timing is layout thrashing due to forced reflow.

Based on what I’m seeing in this trace, I am guessing that the culprit is https://github.com/cypress-io/cypress/pull/6000 or https://github.com/cypress-io/cypress/pull/5916, but I haven’t done a bisect.

[vjau]

I have the same problem. In my case it’s limited to Electron. Version 4 is the same

Running times : 
3.7.0
------ 
Chrome 80
1st run 95.31
2nd run 94.98

Electron 73
1st run 49.4
2nd 60.06
3rd 50.93

3.8.X
------


Chrome 80
1st run 103.15
2nd run 101
3rd run 102.98

Electron 78
1st run 110.90
2nd run 355.23
3nd run doesn't complete