Screen readers not announcing encountered break points

[bjduarte]

@sana-ajani
@isidorn

Issue Type: Bug

Issue:

VoiceOver is not announcing when a break point is encountered within the file.

Expected:

VoiceOver should announce when the cursor encounters a break point within the file.

Actual:

VoiceOver is not announcing when it encounters a break point.

Recreate:

• Turn on VoiceOver
• Insert a break point within a file
• Use the up and down arrow keys to move the cursor over the line containing the break point

VS Code version: Code 1.42.1 (c47d83b, 2020-02-11T14:44:27.652Z)
OS version: Darwin x64 19.3.0

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz (8 x 2700)
GPU Status	2d_canvas: enabled flash_3d: enabled flash_stage3d: enabled flash_stage3d_baseline: enabled gpu_compositing: enabled metal: disabled_off multiple_raster_threads: enabled_on oop_rasterization: disabled_off protected_video_decode: unavailable_off rasterization: enabled skia_renderer: disabled_off surface_control: disabled_off surface_synchronization: enabled_on video_decode: enabled viz_display_compositor: enabled_on viz_hit_test_surface_layer: disabled_off webgl: enabled webgl2: enabled
Load (avg)	2, 1, 2
Memory (System)	16.00GB (3.72GB free)
Process Argv
Screen Reader	yes
VM	0%

Extensions (3)

Extension	Author (truncated)	Version
python	ms-	2020.2.63072
python	tht	0.2.3
vscodeintellicode	Vis	1.2.5

[isidorn]

Using aria alerts to announce that there is a breakpoint on that line seemed too intrusive and potentially spammy. Thus we decided to have F1 > "focus on breakpoints" command that can take the user to the breakpoints view where he / she can navigate over them.

@redeyedragon Would you prefer that on every line we announce that a breakpoint is present? Would you expect that to be announced before or after the line content?

Please note that we announce once breakpoints get hit, which might be related to this.

[bjduarte]

I understand what you are saying it could seem a bit intrusive but here is my perspective. If someone is at the point where they are inserting breakpoints they are probably in debug mode which means they are adding and removing several break points. It is absolutely necessary that the user knows where they have break points placed within the file. I believe it should be announced on the line where the break point is placed and it should be announced at the beginning of the line. As I move through my code I do not want to have to listen to every lines contents to find where I have placed a break point. If I hear it at the start of the line I can decide from there to skip it or read the rest of the line.

I went through the F1 list of commands. There is no key binding set for moving to or through break points. This might help a bit but still there is no indication through speech that a break point is present on a line.

[isidorn]

Ok this makes sense, and thanks for feedback. Assigning to next milestone (March) to investigate further then what could be done here.

As for F1 commands, try to F1 > focus breakpoints view and in there you can jump to breakpoints and navigate which should help.

[isidorn]

After thinking about this a bit furhter I believe this is a subset of a more general issue: items in the glyph margin are not read out to screen readers.
Same as a breakpoint can be on a line, we can also have a code action or a folding or a line number.

Related issues #43262
microsoft/monaco-editor#410

Currently our strategy is not to read any of those and users can manually trigger some of those to be read out.
Maybe a more optimal solution would be one command that could be triggered to read all of the following: breakpoints, code actions, line numbers.
Not on the current plan, thus assigning to On Deck.

fyi @alexdima

[bjduarte]

@isidorn
In my opinion and experience if there are attributes being displayed visually, effecting my code base, or providing me with some kind of information relevant to my development environment I need/want to know about that. For example, when I am in VIM and developing code I turn on line numbers so when I view a stack trace I know where the issue is. When I switch to VIM I can jump to the line quickly but it helps to know what line it is on because the issue is not always on the line referenced. similarly, when there are breakpoints, line breaks, syntax issues, or line numbers; I want to know about them. They might not need to be read out verbose but knowing they are present on a line is information a developer needs to know. I think it could be handled in the same way "hints" are handled when you enter a () into a function. It notifies you there is suggested hints but does not display them.

Additionally, it would be awesome if there could be some kind of indicator for tabs in the beginning of the line. If this is a feature already is there a way to enable it? This is another feature that could be done on command with a key stroke. Perhaps a CTRL + T, could read out "2, or 3 to let me know I am indented 3 levels.

[isidorn]

@redeyedragon what you are saying makes sense.

As for reading indentation we plan to tackle that in the current march milestone as is coveed by this issue #88877

[ajborka]

@isidorn I can see where @redeyedragon is coming from on this point. If I have items in the gutter, then I need to know about them because they change the codebase somehow. Something that might work is to expose those elements and allow screen reader developers to present them to the user. @joanmarie is this possible? In the event that vscode must do all the work, having an accessibility group in settings that determines where the breakpoint is announced would be nice.

[isidorn]

We are actually discussing using aria annotations for breakpoints in VS Code. This should hopefuly be the ideal solution, however this is still work in progress on the Chrome and Orca side.
@aleventhal and @joanmarie would know more.

[joanmarie]

Regarding ARIA Annotations: The basics in Orca already are in place in the latest version of Orca. I just have some additional navigation commands to add, and some refinement to do on the messages. I don't think VSCode's use of ARIA Annotations is blocked by/on Orca, especially not for general announcements.

I also don't think you are blocked on Chrome. I have been using Chrome/Chromium to do the ARIA Annotations support in Orca.

I will let @aleventhal chime in regarding exactly what all he would do in terms of the authoring and user experience.

[aleventhal]

Right, the markup will generally work when VSCode is loaded in a browser, as long as you're using a recent unstable version of Chrome. And it sounds like regarding Orca, that announcements will work, there just won't be any special keys to navigate to additional info pointed to by aria-details. For breakpoints, this probably doesn't matter unless you have conditional breakpoint info that you want to convey. Maybe consider using <mark aria-roledescription="breakpoint">. This is older markup that's not really part of what we call ARIA Annotations. It allows you to provide a localized, more specific name for a role. It works across browsers and screen readers, and has the advantage that when you leave the breakpoint, it should hopefully announce that as well. Some caveats: - I would test to be sure it works as I say. - I may be incorrectly assuming that you'd want it to announce something upon leaving the breakpoint (maybe it's already obvious because you arrow to a new line, so the announcement would just be annoying). Feel free to set up a call with me. I'd be happy to go over the various types of markup and ways of doing things. Aaron

…

On Wed, Mar 25, 2020 at 11:49 AM joanmarie ***@***.***> wrote: Regarding ARIA Annotations: The basics in Orca already are in place in the latest version of Orca. I just have some additional navigation commands to add, and some refinement to do on the messages. I don't think VSCode's use of ARIA Annotations is blocked by/on Orca, especially not for general announcements. I also don't think you are blocked on Chrome. I have been using Chrome/Chromium to do the ARIA Annotations support in Orca. I will let @aleventhal <https://github.com/aleventhal> chime in regarding exactly what all he would do in terms of the authoring and user experience. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#90842 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKQAZVQ5QOCGC2QI3YJ6H3RJIRXZANCNFSM4KWWX5QA> .

[isidorn]

@joanmarie @aleventhal Thanks for providing more details.
Assigning to April to look into this then. Once I try it out I will give more feedback. Thanks

[isidorn]

@aleventhal @joanmarie I have looked and thought a bit about this. Currently VS Code uses a hidden textArea where we put the text content of editor so that it would be accessible. Even though this has some issues with Chrome on macOS this approach works well in general so far.
We already set a role of textBox and aria-roledescription to editor of this textArea. So we do not have a structured HTML object for each line of the text.

Due to that I do not see how we could use the aria-description to present a breakpoint in the editor, since we could only place it on the textArea element and that would be too global (as it has multiple lines).

However I do see some other use cases for aria-roledescription, aria-details and aria-descripton. For example our Breakpoints view which has a structured HTML list strucutre for each breakpoint. And many other of our Tree / List views.

@aleventhal please let me know if I missed something while reading the spec and if you have ideas for better use of this with our textArea editor. I am also open for a call, though please note that I am in the CET timezone. Thanks!

fyi @alexdima

[aleventhal]

Hi. If you switch from using a plain textarea to a contenteditable, you can still do what you are doing but also add any kind of semantics to substrings that you want. Note that either offscreen text editor approach will cause problems with tracking in magnifiers like ZoomText. Google Docs had the same issue, using a contentefitable offscreen.

…

On Fri, Apr 17, 2020, 5:37 AM Isidor Nikolic ***@***.***> wrote: @aleventhal <https://github.com/aleventhal> @joanmarie <https://github.com/joanmarie> I have looked and thought a bit about this. Currently VS Code uses a hidden textArea where we put the text content of editor so that it would be accessible. Even though this has some issues with Chrome on macOS this approach works well in general so far. We already set a role of textBox and aria-roledescription to editor of this textArea. So we do not have a structured HTML object for each line of the text. Due to that I do not see how we could use the aria-description to present a breakpoint in the editor, since we could only place it on the textArea element and that would be too global (as it has multiple lines). However I do see some other use cases for aria-roledescription, aria-details and aria-descripton. For example our Breakpoints view which has a structured HTML list strucutre for each breakpoint. And many other of our Tree / List views. @aleventhal <https://github.com/aleventhal> please let me know if I missed something while reading the spec and if you have ideas for better use of this with our textArea editor. I am also open for a call, though please note that I am in the CET timezone. Thanks! fyi @alexdima <https://github.com/alexdima> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#90842 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKQAZRXGGLHQNB5NYDHBLDRNAPPDANCNFSM4KWWX5QA> .

[isidorn]

Thanks for the suggestion. We can experiment with changing our textarea to contenteditable, however that would require a larger refactoring, thus pushing this out to next milestone.

[aleventhal]

No problem. It's the only way you'll be able to do anything interesting with accessibility, without resorting to live region hacks.

…

On Fri, Apr 24, 2020 at 3:38 AM Isidor Nikolic ***@***.***> wrote: Thanks for the suggestion. We can experiment with changing our textarea to contenteditable, however that would require a larger refactoring, thus pushing this out to next milestone. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#90842 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKQAZW45CJIROILAVWJS7LROE6YXANCNFSM4KWWX5QA> .

[LeonarddeR]

If an editable content approach is used, that might also open up possibilities to fix #90578 and #95428.

[isidorn]

Created this item for the contenteditable exploration which I plan to do this milestone #97154

[cannona]

Hi. I'm also struggling with this issue. Mostly, I just want a notification when I reach a line with folded code.

One option you might consider would be to add earcons. I'm not sure if there is ever any additional information provided along side these left-margin annotations, other than the fact that they exist, but playing a specific sound when the user reaches a line with folded code, or a breakpoint, would work well for me, and is not likely to interfere with the users ability to listen to their screen reader. Unfortunately, this would not help users who are Deaf or hard of hearing, or who, for whatever reason, use braille only.

[isidorn]

Quick update: we are looking into doing quick announcements on the vscode side when a decoration is encountered (like a breakpoint). More details in this issue #130565 (comment)
Feedback welcome.

fyi @hediet

[isidorn]

Fixed via #140044 by @hediet
Sound design by @gino-scarpino
This will be present in VS Code insiders from Monday. We would really appreciate the feedback.
Thank you!

Verifier:

1. Turn on a screen reader, or set "audioCues.enabled": "on"
2. Move cursor to a line with a breakpoint. Make sure there is a "stop-like" sound