Flakey AJAX requests when using Cypress.on('fail') and the test runner

Current behavior:

When using the Cypress.on('fail', function(err, runnable) {}); method I am posting an AJAX request to our application like so:

    Cypress.on('fail', function(err, runnable) {
      // we now have access to the err instance
      // and runnable this failed on
        setTimeout(function() {
          axios.post('https://site.com', {
            error_message: runnable.err.message,
          })
          .then(function (res) {
            console.log(res);
          })
          .catch(function (error) {
            console.log(error);
          });
        }, 0);
        throw err;
    });

Not sure why but runnable won’t work unless I wrap it in a function or timeout, but that is another issue so don’t get confused by it.

When using npx cypress open and then selecting my spec the AJAX request sends every time without any issues.

However if I run my tests using npx cypress run on my CircleCI job the AJAX request is very flakey. Sometimes it sends and sometimes it doesn’t. I’m guessing this is because the test runner is killing the spec before the AJAX request can be sent.

Desired behavior:

I’d like to know how we can stop the spec from being killed until the request has been successfully sent. I’ve already tried using cy.wait but this doesn’t work.

Is there a way that I can manually fail a test in the promise on my AJAX request?

Steps to reproduce:

Run an AJAX request inside of Cypress.on('fail') and then use the npx cypress run command. Alternatively any logic that can take a bit of time within here such as a setTimeout can’t be ran.

Versions (2.1.0, OSX, Chrome)

[nickclicksco]

By the way… are we free to use the API that is used by the Cypress dashboard? CORS is enabled so I’m guessing so. Are there any limitations on the amount of times I can ping the endpoint?

I find an API that returns test failures far more valuable than the actual dashboard in my case.

[jennifer-shehane]

I’d recommend using the Cypress Module API to achieve this today. You can access all failures after the test run and handle them however you want.