Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UI Automation in Windows Console: Use UIA by default on Windows 10 version 2004 and later #10716

Closed
wants to merge 8 commits into from

Conversation

codeofdusk
Copy link
Contributor

Link to issue number:

Roles back #10682 and #10684. Blocked by microsoft/terminal#4018.

Summary of the issue:

microsoft/terminal#4018 provides native word navigation support and fixes to various UIA text range methods. Since the native implementation can now be trusted in more situations, re-enabling UIA in consoles by default on patched Windows releases is strongly worth considering.

Description of how this pull request fixes the issue:

  • Refactor consoleUIATextInfo into consoleUIATextInfo (a base class and the textInfo implementation to use on newer Windows releases) and consoleUIATextInfoPre2004 (containing the older workarounds).
  • Factor out movement and bounds checking into separate _move and move methods, respectively.
  • Use native word navigation and default UIATextInfo movement and comparison methods on Windows 10 2004.
  • Use UIA by default on Windows 10 version 2004 and later.

Testing performed:

Tested with the build provided in the upstream PR. Further testing and feedback from @LeonarddeR and @michaelDCurran would be highly appreciated.

Known issues with pull request:

  • This PR assumes that Refactor UiaTextRange For Improved Navigation and Reliability microsoft/terminal#4018 will be merged in time for Windows 10 version 2004. If not, the version numbers should be changed accordingly to target the Windows 10 release where that PR is merged.
  • WIN10_VERSIONS_TO_BUILDS[2004] is a placeholder value since version 2004 has not yet been released. This build number should be updated for the stable release.

Change log entry:

== New Features ==

@AppVeyorBot
Copy link

See test results for failed build of commit 828d40f7e8

@AppVeyorBot
Copy link

See test results for failed build of commit 1faa8e90b0

@LeonarddeR
Copy link
Collaborator

Is it also within the scope of microsoft/terminal#4018 to address other UIA issues, like:

  • Many blank lines at the end of the console
  • Issues with visible ranges calculation we bumped into earlier

@codeofdusk
Copy link
Contributor Author

codeofdusk commented Jan 24, 2020 via email

@LeonarddeR
Copy link
Collaborator

LeonarddeR commented Jan 24, 2020 via email

@codeofdusk
Copy link
Contributor Author

I don't recall the exact issue nor an issue number.

See #10559.

@codeofdusk
Copy link
Contributor Author

codeofdusk commented Feb 13, 2020

Superseded by #10784.

@codeofdusk codeofdusk closed this Feb 13, 2020
codeofdusk added a commit to codeofdusk/nvda that referenced this pull request Apr 1, 2020
michaelDCurran pushed a commit that referenced this pull request Apr 6, 2020
* Bring over ConsoleUIATextInfo refactoring from #10716.

* Add WinTerminalUIA

* Style.

* Review actions.

* microsoft/terminal#4495: support fixed GetVisibleRanges.

* KeyboardHandlerBasedTypedCharSupport -> EnhancedTermTypedCharSupport, add _shouldUseToUnicodeEx flag and set False for WinTerminalUIA.

* Update doc comments.

* Update copyrights.

* Review actions.

* Factor out bounding into separate method.

* Re-introduce KeyboardHandlerBasedTypedCharSupport.

* Fix UIA console support for newer Windows Console builds.

* Revert copyright header changes.

* Revert another no longer necessary copyright header change.

* Fix comments.

* Update docstring.

* Review actions.

* Update source/NVDAObjects/behaviors.py

Co-Authored-By: Leonard de Ruijter <leonardder@users.noreply.github.com>

* Add class docstring.

* Optimization: clear character buffers when reporting new lines.

This saves us from scanning the old and new lines twice in _calculateNewText (once for diffing and once for finding nonblank indices), and fixes typed word reporting for 21H1 console.

This new approach does not resolve #10942.

* Update inaccurate information in user guide.

Co-authored-by: Leonard de Ruijter <leonardder@users.noreply.github.com>
@codeofdusk codeofdusk deleted the cmduia-default-2004 branch November 4, 2021 22:36
seanbudd pushed a commit that referenced this pull request Jun 21, 2022
…(Windows 11 Sun Valley 2) (#10964)

Supersedes #9771 and #10716. Closes #1682. Closes #8653. Closes #9867. Closes #11172. Closes #11554.

Summary of the issue:

Microsoft has significantly improved performance and reliability of UIA console:
* microsoft/terminal#4018 is an almost complete rewrite of the UIA code which makes the console's UIA implementation more closely align with the API specification.
* microsoft/terminal#10886, microsoft/terminal#10925, and microsoft/terminal#11253 form a robust testing methodology for the UIA implementation, including bug fixes in response to newly added tests based on Word's UIA implementation.
* microsoft/terminal#11122 removes the thousands of empty lines at the end of the console buffer, significantly improving performance and stability. Since all console text ranges are now within the text buffer's bounds, it is no longer possible for console to crash due to the manipulation by UIA clients of an out-of-bounds text range.
* Countless other accessibility-related PRs too numerous to list here.

We should enable UIA support on new Windows Console builds by default for performance improvement and controllable password suppression.

Description of how this pull request fixes the issue:

This PR:
* Exposes all three options for the UIA console feature flag in the UI (replaces the UIA check box with a combo box).
* Adds a runtime check to test if `apiLevel >= FORMATTED`, and use UIA in this case when the user preference is auto. This is the case on Windows 11 Sun Valley 2 (SV2) available now in beta and set for release in the second half of 2022.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants