Pattern for browsable 'locked' or 'answered' quiz/test/exam question?

[brennanyoung]

Use case: e-learning quiz, test or exam where we want to 'lock' an answer in such a way that the user can 'browse back' and be reminded what they answered, but not permit any changes. Essentially a "write-only" form control.

This is very especially in the case where the question remains on the screen after being answered. What changes should be made to the question widget, so that it may be locked, but still browsable?

A pattern is required because the implementation is not obvious, and there are a number of aria attributes which may be 'false friends' for this job.

• A locked answer is non-operable, but neither disabled nor aria-disabled
• A locked answer is non-editable, but not aria-readonly
• A given answer is not aria-checked, even though there may be a cross or checkmark.
• The 'correct' answer (e.g. in a radiogroup) is not necessarily the one that got selected.

What, then, is the optimal approach? It was recently suggested that the optimal thing was to put aria-hidden="true" on the question, and add a brief accessible description (or perhaps aria-details) along the lines of
"Question: 2 Who is the best Beatle? You answered John. The correct answer is George"
Are there other viable implementations?

Can we cook up a working pattern and present it on APG?

[mcking65]

Use case: e-learning quiz, test or exam where we want to 'lock' an answer in such a way that the user can 'browse back' and be reminded what they answered, but not permit any changes. Essentially a "write-only" form control.

What does "write-only" mean? What is "writable" if the user can't change anything?

This is very especially in the case where the question remains on the screen after being answered. What changes should be made to the question widget, so that it may be locked, but still browsable?

A pattern is required because the implementation is not obvious, and there are a number of aria attributes which may be 'false friends' for this job.

• A locked answer is non-operable, but neither disabled nor aria-disabled

If it is non-operable, why is aria-disable not appropriate?

• A locked answer is non-editable, but not aria-readonly

If there is an edit field that can't be edited, how is it not "read only"?

• A given answer is not aria-checked, even though there may be a cross or checkmark.
• The 'correct' answer (e.g. in a radiogroup) is not necessarily the one that got selected.

Yes, aria-checked and aria-selected do not mean "correct". They communicate the user's choice.

What, then, is the optimal approach? It was recently suggested that the optimal thing was to put aria-hidden="true" on the question, and add a brief accessible description (or perhaps aria-details) along the lines of "Question: 2 Who is the best Beatle? You answered John. The correct answer is George" Are there other viable implementations?

Why hide the question with aria-hidden? If it is visible on the screen, then hiding with aria-hidden is problematic.

Is the phrase "You answered John. The correct answer is George" visible on the screen? If not, why not?

Can we cook up a working pattern and present it on APG?

The task force welcomes new proposals. To publish a proposal in the APG, we would need consensus that the pattern illustrates how to solve a common need in a way that is consistent with the ARIA specification. To discuss this with the task force, I think we need more clarity on the requirements and why you would not use some of the attributes you mentioned.

It would likely help to have a description of what is presented to users who do not rely on assistive technologies, perhaps accompanied by a screen shot.

[css-meeting-bot]

The ARIA Authoring Practices (APG) Task Force just discussed Pattern for browsable 'locked' or 'answered' quiz/test/exam question?.

The full IRC log of that discussion

<MarkMcCarthy> TOPIC: Pattern for browsable 'locked' or 'answered' quiz/test/exam question?
<MarkMcCarthy> github: https://github.com//issues/2508
<MarkMcCarthy> Matt_King: might punt this one until we get some more answers
<MarkMcCarthy> Matt_King: we'll check in on this next week as we get more clarification
<MarkMcCarthy> jongund: i think this is pretty relevant in online trainings/courses
<MarkMcCarthy> Matt_King: in my experience, these responses come up in popups, after your answer is submitted. rather than everything being on one page.

[brennanyoung]

Please disregard "write only". I apologise for the misleading turn of phrase. I merely meant that once the question is answered it is no longer operable, but the question, your answer (and any alternative answers) are still visible/browsable on screen.

If it is non-operable, why is aria-disable not appropriate?
If there is an edit field that can't be edited, how is it not "read only"?

Maybe these are appropriate. I was informed that a locked/answered question (in our case, the question is always a radiogroup or checkbox group) was semantically different from an operable/unanswered one.

The problem is that browsing back into the answered question, with reports of disabled or read-only checkboxes or radio groups does not give the same explicit clarity as a simple line of text "you answered X, the correct answer is Y". That meaning must be constructed from a series of fragmentary AT utterances, which are quite likely to be announced 'backwards'

Is the phrase "You answered John. The correct answer is George" visible on the screen? If not, why not?
No. But say "John" was the answer given, and that answer now has a red cross beside it, whereas the non-selected "George" has a checkmark, then that phrase captures explicitly what is expressed through visual means.

It would likely help to have a description of what is presented to users who do not rely on assistive technologies, perhaps accompanied by a screen shot.

I'm including some design mockups of answered questions. These ones do not include the indication of the correct answer Maybe the trick is simply to apply aria-readonly as you suggest, and assume that AT users will make enough sense of the reports about read-only checkboxes, some of which are marked "correct" (and how do we express that?).

Somehow I find this inferior to a simple statement "you answered X, the correct answer is Y"

[css-meeting-bot]

The ARIA Authoring Practices (APG) Task Force just discussed Pattern for browsable 'locked' or 'answered' quiz/test/exam question? #2508.

The full IRC log of that discussion

<MarkMcCarthy> TOPIC: Pattern for browsable 'locked' or 'answered' quiz/test/exam question? #2508
<MarkMcCarthy> github: https://github.com//issues/2508
<MarkMcCarthy> Matt_King: we got responses from Brennan.
<MarkMcCarthy> MarkMcCarthy: [reading Brennan's response]
<MarkMcCarthy> jongund_: sometimes, if "you answered X" type of statement goes after a question, an AT user has to get to that AFTER getting through disabled controls, so putting it before the quesdtion could help
<MarkMcCarthy> jamesn: adding that (before the question) AND -readonly would be perfect
<MarkMcCarthy> Matt_King: ultimately, I think that's good feedback.
<MarkMcCarthy> jongund_: should be it be signified in a landmark or heading? other than plain text, anyway?
<MarkMcCarthy> Matt_King: semantically, if the questions and answers are list items or paragraphs, or otherwise are supported by a heading structure, I wouldn't recommend it
<MarkMcCarthy> jongund_: what about a region with an accname of "Question Result" or something similar?
<MarkMcCarthy> Matt_King: if there's one question per page, and already a few regions, sure. if there's 40 on a page, and it's imperative to get to the nav region, that could clog things up
<MarkMcCarthy> Matt_King: I'm not sure what could be widely reusable for a pattern here. For inclusive tests/quizzes, absolutely. broadly speaking, though, i'm not sure.
<MarkMcCarthy> jamesn: there's definitely a use case, but every test designer may choose soemthing different
<MarkMcCarthy> jongund_: a dialog box could also be a good way to express feedback/answers to questions
<MarkMcCarthy> Matt_King: that works well for us here at Meta
<MarkMcCarthy> jamesn: this is a perfect case for readonly. it's readable by everyone, and not disabled.
<MarkMcCarthy> Matt_King: why?
<MarkMcCarthy> jamesn: the disabled attribute makes it unreadable (visually) - they don't pass contrast. it effectively says "this isn't relevant"
<MarkMcCarthy> jamesn: the fact that HTML doesn't support readonly... it's technicalities, and IMO should be updated
<MarkMcCarthy> jamesn: i've worked on several apps where disabled and readonly both should and can be used, and they mean different things
<MarkMcCarthy> jamesn: we allow aria-disabled and aria-readonly on many different things, though it may or may not be supported by all user agents
<MarkMcCarthy> Matt_King: So, our feedback is that aria-disabled is the most interoperable solution, moreso than HTML disabled.
<MarkMcCarthy> jamesn: Yes, but then you have to make it non-operable in different ways (could even be a good use of a <div> vs. <input>)
<MarkMcCarthy> jamesn: or use <disabled> and override the CSS to make it perceivable
<MarkMcCarthy> Matt_King: just depends on how they want to build it

[mcking65]

@brennanyoung wrote:

Maybe the trick is simply to apply aria-readonly as you suggest, and assume that AT users will make enough sense of the reports about read-only checkboxes, some of which are marked "correct" (and how do we express that?).

The task force recommendation is to use aria-disabled and disable the inputs or use the disabled attribute and override the CSS.

Somehow I find this inferior to a simple statement "you answered X, the correct answer is Y"

Yes, it is inferior. So, the task force also recommends incorporating such statements into the design to clarify the UI for all users.

I hope this feedback is helpful.

The APG Task Force does not have capacity to incorporate it into a pattern at this time. It is a very narrow use case for which there are many design solutions that could be made accessible by utilizing the guidance that is already available in the APG and from other sources.

If you have follow-up questions, feel free to re-open the issue.