New API RFC

Issues with the current API

• Multiple optional positional arguments in the checkA11y() method make it awkward to use and very difficult to extend. We have several open pull requests that are adding a new argument at the end.
• Requires too much setup to have violations summary either in the terminal or JSON that could be supplied to another too.
• Doesn’t support Cypress chaining in the checkA11y() method, and uses a custom context argument. This makes the usage less idiomatic and doesn’t allow integration with tools like Cypress Testing Library.
• No way to define defaults for the checkA11y() method, we have to pass things like violationCallback in every call or create a custom command.

Related issues: #40, #49, #62, #67, #68

New API proposal

1. Remove the context argument of the checkA11y() method, and rely on the Cypress chaining instead:

cy.checkA11y() // Check the whole document
cy.getByTestId('my-modal').checkA11y() // Check part of the document

1. Replace the rest of the arguments with an optional object of the following shape:

interface CypressAxeOptions {
  axeOptions?: axe.RunOptions, // This used to be `options` argument
  shouldFail?: (violations: RunResults[], results: RunResults[]) => boolean,  // Replaces `includedImpacts` and `skipFailures`
  reporters?: Reporter[]
}

type Reporter = (results: RunResults[]) => void

interface RunResults {
  filename: string, // Test file
  label?: string, // Label passed to checkA11y
  results: axe.Result[]
}

The defaults are going to be:

{
  axeOptions: undefined,
  shouldFail: (violations) => violations.length > 0,  // Fail on any number of violations
  reporters: [
   require('cypress-axe/cli-reporter')
  ]
}

Reporters are replacing the violationCallback: they are more flexible, have access to both passes and violations, and you could have global reporters that would run after all tests, or pass a reporter for a separate checkA11y() method call. The default reporter is printing an overall summary after all tests.

1. Add a new optional label to the checkA11y() method:

cy.checkA11y(options?: CypressAxeOptions, label?: string)

Similar to labels describe() or it() methods, to identify results of a particular call.

1. Add a new method to define defaults:

cy.configureCypressAxe(options: CypressAxeOptions)

It should accept the same object as the checkA11y() method, and set default values that could be overwritten in checkA11y().

---

I hope this covers all the use cases we have. I’m not sure that all the names are clear, so if you have any feedback, leave a comment 😉

[kristina-hager]

Well, I suppose reasonable people may disagree on this matter.

However, all the extra functionality this package wraps may be ignored by the user especially if the following changes can be included:

• return the ‘results’ object from the function (this object can be filtered w/ the additional, optional features like ‘minImpact’)
• change the argument skipFailures (optional, defaults to false) to failOnViolations (optional, defaults to false). This way users can opt in to having the assert coded up for them. However, users interested in inspecting the results object themselves and doing their own asserts have an easy path.

In addition, this change would bring this library in line with the majority of other axe-core extensions that do not provide such features and do not do the assert for the user while allowing the additional bells/whistles that some find appealing to remain.

[kyrstenkelly]

I am a fan of this new API proposal. I agree that returning all results is useful in case someone wants to do something more custom, but I also personally appreciate having a nice default reporter so I don’t have to put in extra effort there.