-
-
Notifications
You must be signed in to change notification settings - Fork 680
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support dictionary/data lookups of text #1312
Merged
ychin
merged 1 commit into
macvim-dev:master
from
ychin:text-input-client-support-lookup
Oct 12, 2022
Merged
Support dictionary/data lookups of text #1312
ychin
merged 1 commit into
macvim-dev:master
from
ychin:text-input-client-support-lookup
Oct 12, 2022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ychin
force-pushed
the
text-input-client-support-lookup
branch
from
October 11, 2022 19:47
d6fc1d1
to
c8ccd4e
Compare
ychin
force-pushed
the
text-input-client-support-lookup
branch
from
October 11, 2022 20:11
c8ccd4e
to
2928f77
Compare
9 tasks
This adds support for looking up data under the mouse cursor. Usually it will bring up a dictionary, but other times it could be a Wikipedia article, Siri knowledge, etc. Apple doesn't really have a good name for it, other than "looking up data", "quick look" (a confusingly similar name with the other Quick Look OS feature), or "show definition". You can activate this by doing Ctrl-Cmd-D when the mouse is over a cursor. If you have a trackpad, you can also either activate this using Force click or three-finger tap (depends on your system preference settings). Note that for Force click, this could potentially make it impossible to use the MacVim `<ForceClick>` mapping in Vim, which allows you to map a force click to a Vim command (macvim-dev#716). This is handled by having a new setting (under a new "Input" preference pane which will have more populated later) that allows you to choose whether to use Force click for data lookup or Vim's `<ForceClick>` mapping. If you have configured to use three-finger taps though this setting wouldn't do anything, and `<ForceClick>` is always send to the Vim mapping. Also, this is lacking a lot of features that a normal macOS application would get, e.g. looking up selected texts (e.g. if you have "ice cream", you may want to select the whole thing to look up the phrase, rather than just "ice" or "cream"), data detector, and much more (e.g. custom API support). They will be done later as part of macvim-dev#1311. Technical details below: The way the OS knows how to look up the data and present it is by talking to the NSTextInput/NSTextInputClient. Previously MacVim implemented NSTextInput partially, and didn't implement the critical firstRectForCharacterRange:actualRange and characterIndexForPoint: functions. First, in this change we change from NSTextInput to NSTextInputClient (which is the newer non-deprecated version), and implement those functions, which allows the OS to query the text storage. By default, the OS sends a quickLookWithEvent: call to us whenever the lookup happens but for some odd reason this isn't automatic for Force clicks, presumably because some apps want to handle Force clicks manually (this is why some apps only work for three-finger taps but not Force clicks for lookups). This isn't documented but I found references in iTerm/Firefox, and basically we just need to manually handle it and send off quickLookWithEvent: when handling Force clicks. For implementing the NSTextInputClient properly, the main issue is making sure that can work properly with input methods / marked texts, which is the other primary purpose for this class (other than inputting keys). For data lookups, I'm representing the grid as a row-major text (with no newline/space in between) and expose that to the OS. This already has some issue because it doesn't handle Vim vertical splits well, as MacVim doesn't really have access to detailed Vim text buffers easily (unless we do a lot of calls back-and-forth). This means wrapped texts won't be looked up properly, which I think is ok. Also, the OS APIs deal with UTF-8 indices, so we can't just convert row/column to raw indices and have to do a lot of character length calculations (especially for wide chars like CJK or emojis) to make sure the returned ranges are consistent and valid. For marked texts though, this presents a challenge because Vim doesn't really have a strong enough API to communicate back-and-forth about the marked positions and whatnot (it only let the GUI know where the current cursor is), and it's hard to implement APIs like `markedRange` properly because some marked texts could be hidden or wrapped (if you implement some of these functions improperly Apple's input methods could start misbehaving especially when you use arrow keys to navigate). In the end I kept the original implementation for treating the marked texts as a range starting from 0, *only* when we have marked text. Kind of a hack but this makes sure we work both in marked text mode (i.e. when inputting texts) and when doing lookups. For simplicity I made it so that you can't do data lookups when in marked text mode now. Input method: This change also fixes a quirk in input method as a driveby change. Previously the logic for calculating the rect for where the candidate list was quite broken, but now it's calculated correctly using the desired range and the current cursor position. This matters when say using Japanese IM and using the left/right arrow to jump to different sections of the text. If the desired range is in a wrapped line, the new logic would attempt to pin it to the left-most column of where the cursor is in the range. Data detection: Note that the default implementation is quite bare, and lacks a lot of smart data detection. For example, if you put your mouse over a URL, it won't properly select the whole URL, and addresses and dates for example also won't get grouped together properly. This is because these require additional implementation (e.g. using NSDataDetector) instead of coming "for free", and will be handled later. In fact, Apple's WebKit and NSTextView cheats by calling an internal API framework called "Reveal" (which you can find out by intercepting NSTextView's calls and/or looking at WebKit's source code) which is much more powerful and supports looking up package tracking, airline info, and more, but it's not available to third-party software (that's why Safari's lookup is so much better than Chrome/Firefox's). This isn't tested right now. Future task needs to add XCTest support to properly test this as there are a lot of edge cases involved here. Fix macvim-dev#1191 Part of Epic macvim-dev#1311, which contains other items to be implemented.
ychin
force-pushed
the
text-input-client-support-lookup
branch
from
October 11, 2022 23:52
2928f77
to
83e925e
Compare
ychin
added a commit
that referenced
this pull request
Feb 7, 2023
Updated to Vim 9.0.1276 Features ==================== Dictionary lookup -------------------- You can now use Force Touch or Cmd-Ctrl-D to look up definitions of word under the cursor (or selected text in visual mode). This will also preview URLs, and support data types such as phone numbers and addresses. #1312 #1313 This feature can also be invoked programmatically from VimScript (see `:h macvim-lookup`). #1315 Tool bar / Touch Bar / menu icons -------------------- You can now use SF Symbols for Tool bar and Touch Bar icons, including using different symbol styles such as "palette" or "multicolor". Menu items can now also use the `icon=` syntax to specify icons as well. See `:help macvim-toolbar-icon` for details. #1329 The default tool bar also has updated icons to look similar to SF Symbols used by newer macOS versions. #1214 by @sfsam Window management actions -------------------- There are new `macaction`'s for managing the MacVim window. The new `zoomLeft`/`zoomRight` actions allow you to pin the window to the left/right of the screen, and there are also new actions for interacting with Stage Manager (requires macOS 13+). See `:h macvim-actions` for details. #1330 Pre-release updates / Sparkle 2 -------------------- MacVim now supports pre-release software builds. It's sometimes hard for us to release frequent updates due to the desire to pick a stable upstream Vim version, needing to test the release on multiple OS versions, making sure there aren't half-complete or buggy features, and other reasons. This new feature now allows us to push pre-release beta builds out in a more frequent fashion, which could be useful if there are particular features or fixes that you would like to try out before the next official release. Pre-release builds will be released depending on bug fixes and features instead of a fixed cadence. Do note that these pre-release builds may not be as well-validated and may have half-baked features. If you are using the built-in auto-updater to update MacVim, you can turn this on by going to Advanced settings pane, and enable "Enable pre-release software updates". This feature is only available for macOS 10.13 or above. The auto-updater has also been updated from Sparkle 1.27.1 to 2.3.0 for 10.13+ builds. Legacy (10.9-10.12) builds are still using Sparkle 1. See #1332. New Vim features -------------------- New `smoothscroll` option allows you to scroll through a long wrapped line (using Ctrl-E or mouse wheel) without immediately jumping to the next line. (v9.0.0640) `splitscroll` option has been renamed `splitkeep`, with more flexibility than before. (v9.0.0647) Sound playback on macOS is now supported. You can use `has('sound')` to check. See `help sound` for details. (v9.0.0694) Terminals now support `:confirm` for `:q`, etc, which also means MacVim's Cmd-W will work properly for terminal windows. (v9.0.0710) Virtual text had numerous bugs fixed. General ==================== Legacy build for 10.9 - 10.12 -------------------- Per a previous announcement (#1271), the default MacVim binary will now require macOS 10.13 or above. Users of macOS 10.9 - 10.12 can use a separate "legacy" build which will still be supported. The legacy binary will still have the latest versions of Vim and be supported, but may not have all the latest features (e.g. pre-release builds). If you are using the auto-updater (Sparkle) to update MacVim, it should "just work" and find the best version for you. If you are downloading MacVim from the website, there is also a link to download the legacy version marked for 10.9+ as well. If you download the normal binary marked for 10.13+ from the website, it won't work on these older macOS versions. See #1331. Fixes ==================== CoreText Renderer clipping and rendering bugs -------------------- Unicode characters with multiple composing characters (e.g. "x⃗") will now render correctly. #1172 Texts (e.g. Tibetan, Zalgo texts) that are taller than the line height will no longer be clipped inappropriately. You can use a new setting `MMRendererClipToRow` to re-enable clipping if the tall texts are distracting. #995 / #1356 Tab crash -------------------- Fixed a crash when opening new tabs that seems to only occur in macOS 13 Ventura. #1333 Other bugs -------------------- - Fixed non-native full screen not working well with the notch on newer MacBook's when set to not show menu bar. You can also use `MMNonNativeFullScreenSafeAreaBehavior` to force MacVim to use the notch area as well if you don't mind some content being obscured. Note that the previous release also claimed it fixed this, but because the binary was built against an old macOS SDK (Big Sur), the fix did not work in the binary release. #1261 - Allow "Open untitled window: never" and "After last window closes: Quit MacVim" to be set together again. Added safeguards to make sure doing so won't immediately close the app. #1338 - Edit.Cut / Copy menu items will now be properly disabled when there isn't selected text. #1308 - Fixed potential `:emenu` crash when the menu is associated with an action in a non-valid mode. #1305 - Fixed bug where just bringing up the right-click (or the MacVim→Services) menu would somehow copy the selected texts to the system clipboard. #1300 - Fixed a Japanese input method bug where using left/right arrow to move to a different section of the input text would previously result in the candidate list not showing up at the correct position. #1312 - Fix non-CoreText renderer not handling text styles like strikethrough correctly (note: this renderer has been deprecated for a while and you should not use it). #1296 - This release uses an older sh/bash syntax file because the latest one in Vim has a bug. #1358 Misc ==================== New settings: - "No drop shadows" (Appearance). #1301 - "Treat Ctrl-click as right-click" (Input) (#1326). This was previously configurable via command-line, but now also possible in the settings pane under the new "Input" category. "About MacVim" now reports the version number in a clearer way with clearly specified release number vs Vim version. Known Issues ==================== Printing -------------------- Printing using File→Print or `:hardcopy` is currently not working under macOS 13 Ventura due to its removal of PostScript support in the Preview app. This will be fixed in a later release. See the issue for workarounds. #1347 Scripting ==================== - Scripting languages versions: - Perl is now built against 5.30, up from 5.18. - Ruby is now built against 3.2, up from 3.1. Compatibility ==================== Requires macOS 10.9 or above. (10.9 - 10.12 requires downloading a separate legacy build) Script interfaces have compatibility with these versions: - Lua 5.4 - Perl 5.30 - Python2 2.7 - Python3 3.10 - Ruby 3.2
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds support for looking up data under the mouse cursor. Usually it will bring up a dictionary, but other times it could be a Wikipedia article, Siri knowledge, etc. Apple doesn't really have a good name for it, other than "looking up data", "quick look" (a confusingly similar name with the other Quick Look OS feature), or "show definition". You can activate this by doing Ctrl-Cmd-D when the mouse is over a cursor. If you have a trackpad, you can also either activate this using Force click or three-finger tap (depends on your system preference settings).
Note that for Force click, this could potentially make it impossible to use the MacVim
<ForceClick>
mapping in Vim, which allows you to map a force click to a Vim command (#716). This is handled by having a new setting (under a new "Input" preference pane which will have more populated later) that allows you to choose whether to use Force click for data lookup or Vim's<ForceClick>
mapping. If you have configured to use three-finger taps though this setting wouldn't do anything, and<ForceClick>
is always send to the Vim mapping.Also, this is lacking a lot of features that a normal macOS application would get, e.g. looking up selected texts (e.g. if you have "ice cream", you may want to select the whole thing to look up the phrase, rather than just "ice" or "cream"), data detector, and much more (e.g. custom API support). They will be done later as part of #1311.
Technical details below
The way the OS knows how to look up the data and present it is by talking to the NSTextInput/NSTextInputClient. Previously MacVim implemented NSTextInput partially, and didn't implement the critical
firstRectForCharacterRange:actualRange
andcharacterIndexForPoint:
functions. First, in this change we change from NSTextInput to NSTextInputClient (which is the newer non-deprecated version), and implement those functions, which allows the OS to query the text storage.By default, the OS sends a quickLookWithEvent: call to us whenever the lookup happens but for some odd reason this isn't automatic for Force clicks, presumably because some apps want to handle Force clicks manually (this is why some apps only work for three-finger taps but not Force clicks for lookups). This isn't documented but I found references in iTerm/Firefox, and basically we just need to manually handle it and send off quickLookWithEvent: when handling Force clicks.
For implementing the NSTextInputClient properly, the main issue is making sure that can work properly with input methods / marked texts, which is the other primary purpose for this class (other than inputting keys). For data lookups, I'm representing the grid as a row-major text (with no newline/space in between) and expose that to the OS. This already has some issue because it doesn't handle Vim vertical splits well, as MacVim doesn't really have access to detailed Vim text buffers easily (unless we do a lot of calls back-and-forth). This means wrapped texts won't be looked up properly, which I think is ok. Also, the OS APIs deal with UTF-8 indices, so we can't just convert row/column to raw indices and have to do a lot of character length calculations (especially for wide chars like CJK or emojis) to make sure the returned ranges are consistent and valid. For marked texts though, this presents a challenge because Vim doesn't really have a strong enough API to communicate back-and-forth about the marked positions and whatnot (it only let the GUI know where the current cursor is), and it's hard to implement APIs like
markedRange
properly because some marked texts could be hidden or wrapped (if you implement some of these functions improperly Apple's input methods could start misbehaving especially when you use arrow keys to navigate). In the end I kept the original implementation for treating the marked texts as a range starting from 0, only when we have marked text. Kind of a hack but this makes sure we work both in marked text mode (i.e. when inputting texts) and when doing lookups. For simplicity I made it so that you can't do data lookups when in marked text mode now.Input method
This change also fixes a quirk in input method as a driveby change. Previously the logic for calculating the rect for where the candidate list was quite broken, but now it's calculated correctly using the desired range and the current cursor position. This matters when say using Japanese IM and using the left/right arrow to jump to different sections of the text. If the desired range is in a wrapped line, the new logic would attempt to pin it to the left-most column of where the cursor is in the range.
Data detection
Note that the default implementation is quite bare, and lacks a lot of smart data detection. For example, if you put your mouse over a URL, it won't properly select the whole URL (since the word delineation usually stops at the ":"), and addresses and dates for example also won't get grouped together properly. This is because these require additional implementation (e.g. using NSDataDetector) instead of coming "for free", and will be handled later. In fact, Apple's WebKit and NSTextView cheats by calling an internal API framework called "Reveal" (which you can find out by intercepting NSTextView's calls and/or looking at WebKit's source code) which is much more powerful and supports looking up package tracking, airline info, and more, but it's not available to third-party software (that's why Safari's lookup is so much better than Chrome/Firefox's).
This isn't tested right now. Future task needs to add XCTest support to properly test this as there are a lot of edge cases involved here.
Fix #1191
Part of Epic #1311, which contains other items to be implemented.