No longer fail runs based on recording limit

I can’t see any explanation for this behavior here so I’m going to assume this is a bug or a feature request for something that may have been overlooked.

Current behavior:

Once I’ve reached my recording limits for a month (on the free tier), I get this error:

It looks like this is your first time using Cypress: 3.2.0

[16:10:43]  Verifying Cypress can run /root/.cache/Cypress/3.2.0/Cypress [started]
[16:10:45]  Verifying Cypress can run /root/.cache/Cypress/3.2.0/Cypress [completed]

Opening Cypress...
You've exceeded the limit of private test recordings under your free plan this month. The limit is 500 private test recordings.

To continue recording tests this month you must upgrade your account. Please visit your billing to upgrade to another billing plan.

https://on.cypress.io/dashboard/organizations/<org_id>/billing
error Command failed with exit code 1.

Desired behavior:

I’d prefer that you let my tests still run and execute and just print a warning that the tests aren’t being recorded. That way I can continue to run the tests with --record in CI all month long and just take advantage of the recordings whenever I am within my limit for that month.

I can’t update my CI scripts to know when I’m within the limit or when I’m not so I will always run the risk of getting failing CI runs because I might cross the limit threshold. The only safe thing would be for me to not ever record runs in CI, but I would prefer not to do that. I love your dashboard and I would love to take advantage of it as much as possible.

I understand I’m on the free tier and I’m not exactly your target customer so I understand if you chose to stick with test failures as some way of pushing users to the paid tier. But I don’t have the money for the paid tier on this small side project so I would be forced to stop using the dashboard all together.

Versions

Cypress 3.2.0

Thanks in advance

[jennifer-shehane]

After reading feedback provided from this thread and discussing requirements needed to implement this change with our team, we have decided to change how we handle recording limits being reached for free tier plans - to no longer fail and exit the run. The new behavior will reflect how the recording limit is handled for paid plans, the run will exit normally and viewing data will be limited within the Dashboard views.

We don’t have a clear estimate on when this will be delivered, but we wanted to update everyone to know that there is work planned and to reopen the issue.

Thank you all for providing feedback. We welcome criticism on the product decisions we make and want you all to be successful in testing with Cypress.

[thomasboyt]

I find this to be a total dark pattern - it seems like the only reason this is done is to convince people to convert to paid customers to unbreak their build. A warning is fine, even a loud and pronounced one, but to completely exit tests if you run out of recordings is absurd. I should not have to commit changes to my CI config because I ran out of recordings.

I’m pretty disappointed, since I use Cypress at work and was excited to use it on a side project. I enabled dashboard recordings since I usually stay under 500 tests a month, and figured there wouldn’t be any harm in hitting the limit anyways - and now I see this is not the case 😦

Since Cypress doesn’t have any plan between 0 and 100 dollars a month, and my side project is (a) noncommercial and (b) not under an OSI-approved license, there’s not really any path forward here other than either removing Cypress recordings or using one of these wrapper scripts. I get not wanting to give stuff away for free, and wouldn’t have even minded if y’all just didn’t have a free plan, or only had a timed trial, but I think breaking builds when the free limits are hit is the worst imaginable experience I could have with Cypress.