APG Redesign - Usability Study Plan

[isaacdurazo]

Usability Study Plan

Prototype: http://wai-aria-practices.s3-website-us-east-1.amazonaws.com/

Objectives

1. Determine design inconsistencies and usability problems.
2. Test the effectiveness of the new information architecture.
3. Identify if users can easily find and use specific resources.

Research Questions

1. Are participants able to navigate through the website with the new information architecture?
2. How easily can users find the resource they are looking for with the new central navigation?
3. What issues do participants have when trying to find a particular resource?
4. What is the most likely approach a participant takes when trying to access a pattern example?
5. How discoverable are specific sections on the Fundamentals page?

Logistics

Each session will take 1 hour and will be conducted remotely via Zoom or Google meet depending on the participants’ preference. Sessions will be scheduled for the week of Sept 13, 2021, and participants will be compensated with a $100 Visa gift card for their time.

Participants Profiles

The participants in this study are people with different levels of experience in Accessibility, from a couple of years to more than 12 years in the field. Some spend a few hours a week and others work full time implementing accessible experiences.
The job roles of participants include web accessibility expert/consultant/lead, engineer/developer, designer, UI/UX Engineer, and QA Tester.

Scenarios, Tasks, and Questions

1. You are just starting to build ARIA widgets. Find the resources to understand how to implement keyboard support for them.

   • Task completion: participant finds fundamentals/keyboard-interface/

2. You are looking to implement a tri-state checkbox pattern as part of a design system. Find the best practices to implement that pattern.

   • Task completion: participant finds /patterns/checkbox/ and then /examples/checkbox/checkbox-2/checkbox-2.html

3. You are having issues with NVDA/JAWS announcing an ARIA 1.1 Combobox role properly. Find the Combobox example so you can test it.

   • Task completion: participant finds: /examples/combobox/combobox-select-only.html

4. One of your colleagues has a question about a navigation pattern on your company's website.

   This current pattern uses an unordered list in a nav; if a top-level link has a sub-menu of child links, the top-level link uses aria-expanded and the sub-menu uses aria-hidden for toggling the sub-menu.

   His question is, should this pattern use aria-haspopup in addition to aria-expanded and aria-hidden?

   Look up the documentation about this type of pattern.

   • Task completion: participant finds /patterns/menubutton/ and then /examples/menu-button/menu-button-links.html

5. One of your colleagues is just starting with Accessibility and you would like to point them to the different landmark roles and their examples.

   Look up the documentation about landmark roles.

   • Task completion: participant finds /fundamentals/landmark-regions/#landmark-roles

Post Test Interview Questions

1. How would you describe your overall experience?
2. What, if anything, surprised you about the experience?
3. What, if anything, caused you frustration?
4. What do you think about the website overall?
5. What do you think about how information and features are laid out?
6. What do you wish this website had that it currently lacks?