Retry instead of Fail/Pass on non-200 Status Codes

Is this a Feature or Bug?

Feature

Current behavior:

failOnStatusCode: false will silently swallow errors and skip the test on a per visit() basis.

Desired behavior:

While the current behavior is a valid use case, our use case is that non-200 status codes are sometimes expected (because of how the app is deployed for testing internally) and instead we would like the ability to retry instead of swallow, skip and continue.

So maybe a retryOnStatusCode: true or something (or maybe instead of a Boolean it could take an Integer for number of retries).

cc, @ValerieThoma

[brian-mann]

I see… I think it’s debatable whether we make cy.visit retry on status code errors. I can totally see us doing it for network related events (because these things naturally happen), but if your server actually responds its hard for me to imagine retrying by default is a good idea.

For instance, a visit is not idempotent and there could be side effects of retrying. A server may decide to respond appropriately and do things like set cookies if it wanted to. Retrying again would then potentially cause a different thing to happen on the next request…

I’m leaning towards creating natural retryability only for network errors. For application errors (such as the one you’re experiencing) I believe the easiest thing is to handle them yourself by creating a recursive function. Something like this should work…

Disclaimer: I am writing this code right now, it may not work but hopefully in theory you should understand what it’s doing.

const is500Re = /^5/

const visitWhenStable = (url, options) => {
  function req () {
    return cy.request(url, { failOnStatusCode: false ))
    .then((resp) => {
      if (is500Re.test(resp.status)) {
        return req()      
      }
  
      return cy.visit(url, options)
    })
  }
  
  // kickoff
  return req()
}

[tizmagik]

Yes, that’s fair and makes sense, thanks @brian-mann – will give that a shot on our end and post back if there’s any further issues. Right now we’re planning on “overwriting” the cy.visit() command to transparently check for 200 first so that we don’t have to change our test code much if at all.