Possible regression in 'region' landmark assessment rule

Expectation:

As part of the test suite for Google Chrome’s lighthouse project, an accessibility self-test is performed using axe-core on the HTML report which the application generates.

From manual inspection, the HTML assessed by the test does seem to adhere to the practices documented in the ARIA landmark authoring guidelines.

The test passes as expected when using axe-core 3.4.1.

Actual:

After upgrading to axe-core 3.5.1, the accessibility self-test fails (refs: comment, build logs).

Motivation:

This appears to be a false-positive and that’s the main motivation for raising the issue. In addition it temporarily blocks lighthouse from upgrading to axe-core 3.5.1 (or would require a reduction in lighthouse’s test coverage in order to upgrade).

axe-core version: 3.5.1

Browser and Assistive Technology versions

For Tooling issues:
- Node version: v10.15.2 (developer environment), v10.18.1 (AppVeyor environment), v10.19.0 (Travis-CI environment)
- Platform:  Linux (developer environment), Windows (AppVeyor environment), Linux (Travis-CI environment)

[straker]

The <footer> element just prevents .lh-container from being reported as the source element for the error. Nothing too much to look into.

As for the selector, the only reason it was changed was to avoid looking at and head elements since they are never checked for regions. I don’t have anything against changing it back to html, but I’d like to know why the lighthouse test is failing first.

Would it be possible to capture the output of the test HTML output before the assert is run on it? I’d like to see what exactly axe is running against (I know it should be what we’ve already looked at, but that should be passing).

[jayaddison]

Thanks again @straker - yep, after making some small adjustments (particularly this one to ensure that content rendered for the lighthouse self-test is contained by a main element), the region rule seems to assess the output correctly.

In other words, this does seem like a true positive and axe-core seems correct. My only thought regarding why this appeared is that perhaps the html selector in the lighthouse test was not working as expected in the past?

Either way, this issue seems resolved so I’ll go ahead and close it.