Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows 11 emoji panel/navigation menu item: handle UIA elemenet selected event by ignoring it or manipulating NVDA's focus object #16555

Merged
merged 13 commits into from
May 17, 2024
Merged

Windows 11 emoji panel/navigation menu item: handle UIA elemenet selected event by ignoring it or manipulating NVDA's focus object #16555

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
872e1aa
Modern keyboard: define an overlay class for Windows 11 emoji panel n…
josephsl May 15, 2024
86e16e8
Emoji panel/choose overlay classes: recognize list items with Automat…
josephsl May 15, 2024
88ac092
Emoji panel/UIA element selected: let navigation menu item overlay cl…
josephsl May 15, 2024
2843b36
Windows 11 emoji panel/navigation menu item: ignore element selected …
josephsl May 16, 2024
813faab
Windows 11 emoji panel/navigation menu item: report the selected menu…
josephsl May 16, 2024
3917efe
Windows 11 emoji panel/navigation menu item: manipulate focus when em…
josephsl May 16, 2024
a4b9bc1
Winodws 11 emoji panel/navigation menu item: ignore element selected …
josephsl May 16, 2024
c203165
Windows 11 emoji panel/navigation menu item: ignore the event when re…
josephsl May 16, 2024
fc7d87c
Windows 11 emoji panel/navigation menu item: move NVDA focus to syste…
josephsl May 16, 2024
87d09e5
Windows 11 emoji panel/overlay class chooser (Flake8): add missing co…
josephsl May 16, 2024
201ca94
Changelog: add Windows 11 emoji panel navigation menu entries
josephsl May 16, 2024
ab21080
Changelog: clarity (suggested by Code Rabbit AI)
josephsl May 16, 2024
7994efe
Windows 11 emoji panel/navigation menu item: add type hint for elemen…
josephsl May 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 59 additions & 17 deletions source/appModules/windowsinternal_composableshell_experiences_textinput_inputapp.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
import config
import winVersion
import controlTypes
from NVDAObjects.UIA import UIA, XamlEditableText
from NVDAObjects.UIA import UIA, XamlEditableText, ListItem
from NVDAObjects.behaviors import CandidateItem as CandidateItemBehavior, EditableTextWithAutoSelectDetection
from NVDAObjects import NVDAObject

Expand Down Expand Up @@ -128,6 +128,40 @@ def event_UIA_elementSelected(self):
self.reportFocus()


class NavigationMenuItem(ListItem):
"""
A Windows 11 emoji panel navigation menu item.
In Windows 10 Version 1903 and later, emoji panel can be used to insert emojis, kaomojis, and symbols.
System focus cannot move to these choices in Windows 10 but can do so in Windows 11.
In addition to the choices above, Windows 11 adds GIF and clipboard history to navigation menu.
"""
seanbudd marked this conversation as resolved.
Show resolved Hide resolved

def event_UIA_elementSelected(self):
# Workarounds for Windows 11 emoji panel category items.
# Ignore the event altogether.
if (
# #16346: system focus restored.
(focus := api.getFocusObject()).appModule != self.appModule
# #16532: repeat announcement due to pending gain focus event on category entries.
or eventHandler.isPendingEvents("gainFocus")
# #16533: system focus is located in GIF/kaomoji/symbol entry.
or focus.UIAAutomationId.startswith("item-")
):
return
# Manipulate NVDA's focus object.
if (
# #16346: NVDA is stuck in a nonexistent edit field (location is None).
not any(focus.location)
# #16347: focus is once again stuck in top-level modern keyboard window
# after switching to clipboard history from other emoji panel screens.
or focus.firstChild and focus.firstChild.UIAAutomationId == "Windows.Shell.InputApp.FloatingSuggestionUI"
):
eventHandler.queueEvent("gainFocus", self.objectWithFocus())
return
# Report the selected navigation menu item.
super().event_UIA_elementSelected()


class AppModule(appModuleHandler.AppModule):

# Cache the most recently selected item.
Expand All @@ -138,9 +172,14 @@ class AppModule(appModuleHandler.AppModule):
disableBrowseModeByDefault: bool = True

def event_UIA_elementSelected(self, obj, nextHandler):
# Logic for IME candidate items is handled all within its own object
# Logic for the following items is handled by overlay classes
# Therefore pass these events straight on.
if isinstance(obj, ImeCandidateItem):
if isinstance(
obj, (
ImeCandidateItem, # IME candidate items
NavigationMenuItem # Windows 11 emoji panel navigation menu items
)
):
return nextHandler()
# #7273: When this is fired on categories,
# the first emoji from the new category is selected but not announced.
Expand Down Expand Up @@ -353,21 +392,24 @@ def event_gainFocus(self, obj: NVDAObject, nextHandler: Callable[[], None]):

def chooseNVDAObjectOverlayClasses(self, obj, clsList):
if isinstance(obj, UIA):
if obj.role == controlTypes.Role.LISTITEM and (
(
obj.parent.UIAAutomationId in (
"ExpandedCandidateList",
"TEMPLATE_PART_AdaptiveSuggestionList",
if obj.role == controlTypes.Role.LISTITEM:
if (
(
obj.parent.UIAAutomationId in (
"ExpandedCandidateList",
"TEMPLATE_PART_AdaptiveSuggestionList",
)
and obj.parent.parent.UIAAutomationId == "IME_Candidate_Window"
)
and obj.parent.parent.UIAAutomationId == "IME_Candidate_Window"
)
or obj.parent.UIAAutomationId in (
"IME_Candidate_Window",
"IME_Prediction_Window",
"TEMPLATE_PART_CandidatePanel",
)
):
clsList.insert(0, ImeCandidateItem)
or obj.parent.UIAAutomationId in (
"IME_Candidate_Window",
"IME_Prediction_Window",
"TEMPLATE_PART_CandidatePanel",
)
):
clsList.insert(0, ImeCandidateItem)
elif obj.UIAAutomationId.startswith("navigation-menu-item"):
clsList.insert(0, NavigationMenuItem)
elif (
obj.role in (controlTypes.Role.PANE, controlTypes.Role.LIST, controlTypes.Role.POPUPMENU)
and obj.UIAAutomationId in (
Expand Down
4 changes: 3 additions & 1 deletion user_docs/en/changes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,10 @@
* Improved reliability of automatic text readout, particularly in terminal applications. (#15850, #16027, @Danstiv)
* NVDA will correctly announce selection changes when editing a cell's text in Microsoft Excel. (#15843)
* Windows 11 fixes:
* In emoji panel, NVDA will no longer appear to get stuck when closing clipboard history. (#16347, @josephsl)
* In emoji panel, NVDA will no longer appear to get stuck when closing clipboard history and emoji panel. (#16346, #16347, @josephsl)
* NVDA will once again announce visible candidates when opening Windows 11 IME interface. (#14023, @josephsl)
* NVDA will no longer announce "clipboard history" twice when reviewing emoji panel navigation menu items. (#16532, @josephsl)
* NVDA will no longer cut off speech and braille while reviewing kaomojis and symbols in emoji panel. (#16533, @josephsl)
Copy link
Contributor

@coderabbitai coderabbitai bot May 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure clarity and consistency in documentation.

The documentation changes accurately reflect the improvements and fixes mentioned in the PR summary. However, consider rephrasing for better clarity and consistency:

-  * In emoji panel, NVDA will no longer appear to get stuck when closing clipboard history and emoji panel. (#16346, #16347, @josephsl)
+  * NVDA will no longer get stuck when closing the clipboard history and emoji panel in Windows 11. (#16346, #16347, @josephsl)
-  * NVDA will once again announce visible candidates when opening Windows 11 IME interface. (#14023, @josephsl)
+  * NVDA will announce visible candidates again when opening the Windows 11 IME interface. (#14023, @josephsl)
-  * NVDA will no longer announce "clipboard history" twice when reviewing emoji panel navigation menu items. (#16532, @josephsl)
+  * NVDA will no longer announce "clipboard history" twice when navigating through the emoji panel menu items. (#16532, @josephsl)
-  * NVDA will no longer cut off speech and braille while reviewing kaomojis and symbols in emoji panel. (#16533, @josephsl)
+  * NVDA will no longer cut off speech and braille when reviewing kaomojis and symbols in the emoji panel. (#16533, @josephsl)
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
* In emoji panel, NVDA will no longer appear to get stuck when closing clipboard history and emoji panel. (#16346, #16347, @josephsl)
* NVDA will once again announce visible candidates when opening Windows 11 IME interface. (#14023, @josephsl)
* NVDA will no longer announce "clipboard history" twice when reviewing emoji panel navigation menu items. (#16532, @josephsl)
* NVDA will no longer cut off speech and braille while reviewing kaomojis and symbols in emoji panel. (#16533, @josephsl)
* NVDA will no longer get stuck when closing the clipboard history and emoji panel in Windows 11. (#16346, #16347, @josephsl)
* NVDA will announce visible candidates again when opening the Windows 11 IME interface. (#14023, @josephsl)
* NVDA will no longer announce "clipboard history" twice when navigating through the emoji panel menu items. (#16532, @josephsl)
* NVDA will no longer cut off speech and braille when reviewing kaomojis and symbols in the emoji panel. (#16533, @josephsl)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these suggestions look good to me, except perhaps the first one? The first one appears to be changing meaning

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seanbudd, thank you for your feedback. Upon reviewing the first suggestion, I agree that it might change the meaning slightly. The original text emphasizes that NVDA appears to get stuck, while the suggested change implies that it definitively gets stuck. To maintain the original intent, we can rephrase it for clarity without altering the meaning:

-  * In emoji panel, NVDA will no longer appear to get stuck when closing clipboard history and emoji panel. (#16346, #16347, @josephsl)
+  * NVDA will no longer appear to get stuck when closing the clipboard history and emoji panel in Windows 11. (#16346, #16347, @josephsl)

Would you like me to proceed with this revision?

New Learnings Added 
User: seanbudd
PR: nvaccess/nvda#16555
File: user_docs/en/changes.md:27-30
Timestamp: 2024-05-16T01:53:50.659Z
Learning: When suggesting documentation changes, ensure that the rephrased text does not alter the original meaning, especially when the original text includes qualifiers like "appear to."

Note: Learnings are effective only in the context of specific code segments. To apply general coding guidelines, please configure review instructions.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to NV Access and the community: the conversations I'm having with Code Rabbit is not meant for human consumption but feel free to look at it (it also shows how I'm approaching the bot as a software engineer and a grad student/researcher interested in communication and tech):

I see. Two things:

  1. Not all members of the project are trained in technical communication (I am schooled in it), and for many, English is not our first language (me included). So issues on clarity is acceptable (we have a staff member at NV Access who can go over grammar and other issues later).
  2. Look at the unindented line which says "Windows 11 fixes", so there is no need to repeat this in indented lines.

I know you are using sophisticated networks to analyze suggestions based on whatever software development data (philosophy, experiences, documentation, and what not). Do remember his though: whatever was set by the organization (including its discourse and mindset) can and do override commonly held ideas represented by the data you are using. At least, for the first try, you got it right.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!


### Changes for Developers

Expand Down