Tests in Browser

Context:

• We have been dreading this problem for years (see #4201). But it’s time we do the tests in browser.

• Stuart (@bouncey) and Oliver (@ojeytonwilliams) have laid the initial foundation by integrating Cypress(see #37053). This was not available in our earlier days, and has been quite a popular tool for automation testing.

• Alongside Cypress (which uses puppeteer which in turn enables you to work with headless chrome) we can now do browser tests.

Goal (each requirement below can be a PR of its own)

• Create a test suite that can run against all challenges (only for certifications that we offer).
• Create and add suites for testing user workflows.
  • Sign up / Sign in
  • Email change / update
  • Changes to profile data
  • a11y tests and lighthouse tests?
• Create and add suites for testing navigations and layouts across devices.
• Update documentation about e2e tests
• … add more.

Nice to have:

• Challenge tests could be run only for changes on PRs and all tests when on master?

[raisedadead]

We would still be using both in some form. Keep in mind that we would have a combination of both because, we would need both integration tests (also called unit tests), and user acceptance tests (also called e2e tests).

At the minute all the challenge tests are unit tests which are run at build time, but we would also want to be able to re-run the same tests in browser using our test runner, as e2e tests to identify broken functionalities like a real user would.

[Twaha-Rahman]

Ok… I will look into Cypress when I have some time. Thanks for the quick response!