Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to test contrast of nodes scrolled out of view #3772

Closed
WilcoFiers opened this issue Nov 8, 2022 · 7 comments
Closed

Unable to test contrast of nodes scrolled out of view #3772

WilcoFiers opened this issue Nov 8, 2022 · 7 comments
Labels
color contrast Color contrast issues fix Bug fixes pr A pr has been created for the issue target-size
Milestone
axe-core 4.6

Comments

@WilcoFiers
Copy link
Contributor

This problem was introduced in axe-core 4.5. When an element is scrolled out of view, it is not included in the grid, which results (among other things) that it is reported as incomplete once its scrolled out of view.

Here's how to reproduce the problem. These logs should report the same, but the second one reports incomplete instead of a violations.

<h1 style="color: #ABC">Hello World</h1>
<main style="background: #CDF; height: 800px;"></main>
<script src="/axe.js"></script>
<script>
  window.onload = async () => {
    window.scrollTo(0, 0);
    const res0 =  await axe.run({ runOnly: 'color-contrast' })
    console.log({ violations: res0.violations, incomplete: res0.incomplete })

    window.scrollTo(0, 1000);
    const res1 =  await axe.run({ runOnly: 'color-contrast' })
    console.log({ violations: res1.violations, incomplete: res1.incomplete })
  }
</script>
@WilcoFiers WilcoFiers added fix Bug fixes color contrast Color contrast issues target-size labels Nov 8, 2022
@dbjorge
Copy link
Contributor

dbjorge commented Nov 8, 2022

In case it helps with prioritization/sense of scale of the issue: We were just doing some validation of axe-core 4.5.0 in preparation for upgrading Accessibility Insights for Web and found that 4.5.0 detected about 90% fewer color-contrast issues in our big corpus of test sites compared to 4.4.1 (78 vs 836 issues detected). Based on spot checking the cases with different results I suspect this is probably the root cause, but I didn't verify every case.

@WilcoFiers WilcoFiers mentioned this issue Nov 9, 2022
Merged
@straker straker added the pr A pr has been created for the issue label Nov 14, 2022
@straker straker added this to the axe-core 4.5.2 milestone Nov 14, 2022
@straker
Copy link
Contributor

straker commented Nov 17, 2022

@dbjorge axe-core 4.5.2 was released and should fix this issue. Would you be able to pull it down and test it to see if it returns results as expected?

@dbjorge
Copy link
Contributor

dbjorge commented Nov 18, 2022

Yes, I'll rerun with 4.5.2 against our big batch of test sites overnight tonight!

@dbjorge
Copy link
Contributor

dbjorge commented Nov 19, 2022

Yes, that resolved most of the differences.

I did find one class of failure that was reported in 4.4.1 but still not in 4.5.2. It involves a specific type of carousel control currently used in a few Skype marketing pages. An example is the bottom of the main content in https://www.skype.com/en/content-creators/ (I tried to make a more minimal repro but gave up after 5 minutes). This specific control has a few reported color-contrast issues where it renders some carousel items at opacity: .2 as they transition onto the page. axe-core 4.4.1 and 4.5.2 both report those cases for which the content is visible on screen in this state as color-contrast failures. However, 4.4.1 additionally reports the items which are completely off-screen in this state as having color-contrast failures, where 4.5.2 does not report color-contrast failures for the off-screen items.

I think it'd be pretty questionable to call this case an axe-core bug - the content in question is completely off-screen inside a container which uses overflow: visible styling which suppresses the browser's default scrolling behavior, and I don't think it's reasonable to expect axe-core to infer whether there's some custom means of scrolling the elements into view (as the carousel buttons provide in this case). If anything, I think the 4.5.2 behavior of not reporting contrast issues with these specific off-screen elements is probably more reasonable than the 4.4.1 behavior.

@WilcoFiers
Copy link
Contributor Author

@dbjorge this is expected. @straker made a number of improvement that allowed us to better detect when things are off screen / clipped. It sounds like that's what's going on here.

@WilcoFiers WilcoFiers removed this from the axe-core 4.5.3 milestone Nov 21, 2022
@padmavemulapati
Copy link

Still seeing the issue, even when validated with the latest axe-core develop branch code base,
attaching the screen recording here,

Screen.Recording.2022-11-23.at.4.54.29.PM.mov

@straker , can you please provide your observations here on this

@WilcoFiers WilcoFiers added this to the axe-core 4.6 milestone Nov 30, 2022
@padmavemulapati
Copy link

Validated with the latest code base of axe-core develop branch,
test-snippet:

<!DOCTYPE html>
<html lang="en">
  <title>Axe Playground</title>
  <!-- The playground is used for quick local host axe-core during development -->

  <main>
    <h1>Hello World</h1>
    <p style="color: lightblue;">hell world</p>
    <div style="height: 1000vh;"></div>
    <button>end of line</button>
  </main>

  <script src="/axe.js"></script>
  <script src="/test/testutils.js"></script>
  <script>
    axe.testUtils.awaitNestedLoad(
      window,
      () => {
        axe.run(
          {
            runOnly: 'color-contrast',
            elementRef: true
          },
          (err, results) => {
            console.log(err || results);
          }
        );
      },
      err => console.error(err)
    );
  </script>
</html>

even after scrolled, color-contrast rule failing in this situation, which is expected

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
color contrast Color contrast issues fix Bug fixes pr A pr has been created for the issue target-size
Projects
None yet
Milestone
axe-core 4.6
Development

No branches or pull requests
4 participants
@dbjorge @WilcoFiers @straker @padmavemulapati