Jaws screen reader

[gomaltao]

Windows Terminal seems to be totally inaccessible with the JAWS screen reader.

[zadjii-msft]

Which version of the Terminal? Which OS version? Which JAWS version? I'm pretty sure JAWS is supposed to work... /cc @carlos-zamora

[carlos-zamora]

@codeofdusk I know you use NVDA a lot, have you tried using JAWS? Or have you heard from any JAWS users trying to use Windows Terminal at all?

[codeofdusk]

Sorry, no.

@Grahamwp What happens if you try and run wt from the run dialog on your machine? Is the opened terminal accessible at all?

[gomaltao]

Windows Terminal 1.11.2921.0
OS Name: Microsoft Windows 10 Pro
OS Version: 10.0.19042 N/A Build 19042
JAWS Version 2020.2012.13 ILM

I have tried to use WT since the first beta without any success with JAWS.

[gomaltao]

I closed the issue by misstake.

[Grahamwp]

I can confirm this is the case with Windows Terminal version 1.11.2921.0and the latest version of JAWS, 2022.2110.36.400.

[zadjii-msft]

I'm kinda shocked that JAWS doesn't work at all. We should fix that. Does the vintage console host work? If it does, then it shouldn't be too hard to find the piece of console code we're missing in the Terminal that makes JAWS work.

[codeofdusk]

This doesn't surprise me as i highly suspect that JAWS uses the legacy APIs. (only Narrator uses UIA by default at this point, though I'm pushing for NV Access to do so in builds of conhost with the #6986 fix). NVDA exclusively uses UIA in wt though.

[Grahamwp]

Late reply (I tried to reply by email earlier but it didn't work): Assuming you mean cmd.exe, yes, JAWS does work in that. It also workes in Power Shell.

[zadjii-msft]

@codeofdusk Is there any way for us to confirm your suspicion, that they use the legacy APIs? It might be time for us to encourage them to move on to the newer APIs 😄 I'm not sure they have a public issue tracker anywhere...

[gomaltao]

As a senior .net developer I need to work in powershell a lot. Windows terminal would improve my working conditions a lot. I would really like to know where the problem is. JAWS has problem with announcing breakpoints in Visual Studio for several generations of VS ide. Again it would be good to know if the problem is with the screen reader or VS ide.

[codeofdusk]

Maybe @carlos-zamora could do a build of conhost that shows dummy text in the UIA buffer in place of the actual text. If we get his text then they're using UIA, else they're using legacy. (for whatever it's worth I highly doubt they're using UIA for much if at all).

I haven't been an active JAWS user for about ten years, but @Grahamwp can try builds out for us and report back. I also suspect that they won't be amenable to making changes quickly if at all.

[carlos-zamora]

Maybe @carlos-zamora could do a build of conhost...

This issue is tracking Windows Terminal, not conhost. Like you said earlier, since Windows Terminal only has UIA (as in, there's no legacy API option), a screen reader needs to use UIA here.

When I was developing Windows Terminal's UIA patterns, NVDA and Narrator both "worked". Like, they were testable and I could clearly see when I made a mistake. I'd be surprised if JAWS saw Windows Terminal and decided to do nothing (because in its eyes, it would see that it's a random app that exposes UIA, and I assume like any screen reader, it would leverage that and try to use it).

I'll get in contact with JAWS and see what we can do, but I'm pretty sure the issue here is on their end. I'll report back when I get more news 🙂

[codeofdusk]

@carlos-zamora that's a zipped lnk pointing at D:\Terminal\bin\x64\Release\OpenConsole.exe.

[carlos-zamora]

Derp... here's the actual exe.
ConHost - UIA with fake data.zip

[codeofdusk]

For some reason, Windows Defender is flagging that file and I can't figure out how to allow it.

[carlos-zamora]

Ok, try this one then. It's signed.
conhost-x64-release.zip

[codeofdusk]

Thanks @carlos-zamora .

@Grahamwp please do the following on your machine, ping me on Discord if you run into trouble:

1. Download the last build from @carlos-zamora above (this one).
2. Unzip and run OpenConsole.exe.
3. Try to review the window by line.

Do you get normal output (Microsoft Windows blahblah and a prompt) or "fake data from UIA" over and over again? Or something else?

[codeofdusk]

Also try autoread and the virtual buffer if JAWS still has that.

[Grahamwp]

I downloaded the file and I just get the normal Windows console output, whether I use autoread, review the window with the JAWS cursor, or try the virtual buffer.

[codeofdusk]

As expected, they aren't using UIA at all in consoles, so supporting terminal will be a big task.

[zadjii-msft]

Well this is good to have some confirmation and get this loop closed! JAWS is using the legacy accessibility APIs, and they're going to need to move to the modern ones to make the Terminal work with JAWS. UIA has been around for some 15 years now, so I'm a little surprised they haven't added support by now.

Thanks all!

notes found while writing: Microsoft Active Accessibility and UI Automation Compared

[codeofdusk]

Jaws does support UIA in some apps, but not conhost (similar to NVDA's default behaviour before nvaccess/nvda#10964).

Maybe @dglee42 or @RobGallo might know how to attach JAWS terminal support to UIA events via a script (if it even works that way)?

[Grahamwp]

There's also relevant documentation from Freedom Scientific but I wouldn't personally be able to further help with this on the JAWS scripting side.

[carlos-zamora]

FYI, I've managed to reach out to JAWS. Hoping I can provide some direct support to them. They are definitely aware of this issue and I've even forwarded them this GitHub issue. 🙂

[zadjii-msft]

Moving this into 1.14 based on recent conversations.

[zadjii-msft]

@carlos-zamora should have fixed this in #12358, and confirmed that JAWS made the fix on their end as well. We're not sure exactly which version of JAWS the fix is in, but it should be in the next version of the Terminal.