Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support dictionary/data lookups of text #1312

Merged
merged 1 commit into from
Oct 12, 2022

Conversation

ychin
Copy link
Member

@ychin ychin commented Oct 11, 2022

This adds support for looking up data under the mouse cursor. Usually it will bring up a dictionary, but other times it could be a Wikipedia article, Siri knowledge, etc. Apple doesn't really have a good name for it, other than "looking up data", "quick look" (a confusingly similar name with the other Quick Look OS feature), or "show definition". You can activate this by doing Ctrl-Cmd-D when the mouse is over a cursor. If you have a trackpad, you can also either activate this using Force click or three-finger tap (depends on your system preference settings).

Note that for Force click, this could potentially make it impossible to use the MacVim <ForceClick> mapping in Vim, which allows you to map a force click to a Vim command (#716). This is handled by having a new setting (under a new "Input" preference pane which will have more populated later) that allows you to choose whether to use Force click for data lookup or Vim's <ForceClick> mapping. If you have configured to use three-finger taps though this setting wouldn't do anything, and <ForceClick> is always send to the Vim mapping.

Also, this is lacking a lot of features that a normal macOS application would get, e.g. looking up selected texts (e.g. if you have "ice cream", you may want to select the whole thing to look up the phrase, rather than just "ice" or "cream"), data detector, and much more (e.g. custom API support). They will be done later as part of #1311.

Technical details below

The way the OS knows how to look up the data and present it is by talking to the NSTextInput/NSTextInputClient. Previously MacVim implemented NSTextInput partially, and didn't implement the critical firstRectForCharacterRange:actualRange and characterIndexForPoint: functions. First, in this change we change from NSTextInput to NSTextInputClient (which is the newer non-deprecated version), and implement those functions, which allows the OS to query the text storage.

By default, the OS sends a quickLookWithEvent: call to us whenever the lookup happens but for some odd reason this isn't automatic for Force clicks, presumably because some apps want to handle Force clicks manually (this is why some apps only work for three-finger taps but not Force clicks for lookups). This isn't documented but I found references in iTerm/Firefox, and basically we just need to manually handle it and send off quickLookWithEvent: when handling Force clicks.

For implementing the NSTextInputClient properly, the main issue is making sure that can work properly with input methods / marked texts, which is the other primary purpose for this class (other than inputting keys). For data lookups, I'm representing the grid as a row-major text (with no newline/space in between) and expose that to the OS. This already has some issue because it doesn't handle Vim vertical splits well, as MacVim doesn't really have access to detailed Vim text buffers easily (unless we do a lot of calls back-and-forth). This means wrapped texts won't be looked up properly, which I think is ok. Also, the OS APIs deal with UTF-8 indices, so we can't just convert row/column to raw indices and have to do a lot of character length calculations (especially for wide chars like CJK or emojis) to make sure the returned ranges are consistent and valid. For marked texts though, this presents a challenge because Vim doesn't really have a strong enough API to communicate back-and-forth about the marked positions and whatnot (it only let the GUI know where the current cursor is), and it's hard to implement APIs like markedRange properly because some marked texts could be hidden or wrapped (if you implement some of these functions improperly Apple's input methods could start misbehaving especially when you use arrow keys to navigate). In the end I kept the original implementation for treating the marked texts as a range starting from 0, only when we have marked text. Kind of a hack but this makes sure we work both in marked text mode (i.e. when inputting texts) and when doing lookups. For simplicity I made it so that you can't do data lookups when in marked text mode now.

Input method

This change also fixes a quirk in input method as a driveby change. Previously the logic for calculating the rect for where the candidate list was quite broken, but now it's calculated correctly using the desired range and the current cursor position. This matters when say using Japanese IM and using the left/right arrow to jump to different sections of the text. If the desired range is in a wrapped line, the new logic would attempt to pin it to the left-most column of where the cursor is in the range.

Data detection

Note that the default implementation is quite bare, and lacks a lot of smart data detection. For example, if you put your mouse over a URL, it won't properly select the whole URL (since the word delineation usually stops at the ":"), and addresses and dates for example also won't get grouped together properly. This is because these require additional implementation (e.g. using NSDataDetector) instead of coming "for free", and will be handled later. In fact, Apple's WebKit and NSTextView cheats by calling an internal API framework called "Reveal" (which you can find out by intercepting NSTextView's calls and/or looking at WebKit's source code) which is much more powerful and supports looking up package tracking, airline info, and more, but it's not available to third-party software (that's why Safari's lookup is so much better than Chrome/Firefox's).


This isn't tested right now. Future task needs to add XCTest support to properly test this as there are a lot of edge cases involved here.


Fix #1191
Part of Epic #1311, which contains other items to be implemented.

@ychin ychin force-pushed the text-input-client-support-lookup branch from d6fc1d1 to c8ccd4e Compare October 11, 2022 19:47
@ychin
Copy link
Member Author

ychin commented Oct 11, 2022

Screenshot:

image

@ychin ychin force-pushed the text-input-client-support-lookup branch from c8ccd4e to 2928f77 Compare October 11, 2022 20:11
@ychin ychin added this to the Release 175 milestone Oct 11, 2022
This adds support for looking up data under the mouse cursor. Usually it
will bring up a dictionary, but other times it could be a Wikipedia
article, Siri knowledge, etc. Apple doesn't really have a good name for
it, other than "looking up data", "quick look" (a confusingly similar
name with the other Quick Look OS feature), or "show definition". You
can activate this by doing Ctrl-Cmd-D when the mouse is over a cursor.
If you have a trackpad, you can also either activate this using Force
click or three-finger tap (depends on your system preference settings).

Note that for Force click, this could potentially make it impossible to
use the MacVim `<ForceClick>` mapping in Vim, which allows you to map a
force click to a Vim command (macvim-dev#716). This is handled by having a new
setting (under a new "Input" preference pane which will have more
populated later) that allows you to choose whether to use Force click
for data lookup or Vim's `<ForceClick>` mapping. If you have configured
to use three-finger taps though this setting wouldn't do anything, and
`<ForceClick>` is always send to the Vim mapping.

Also, this is lacking a lot of features that a normal macOS application
would get, e.g. looking up selected texts (e.g. if you have "ice cream",
you may want to select the whole thing to look up the phrase, rather
than just "ice" or "cream"), data detector, and much more (e.g. custom
API support). They will be done later as part of macvim-dev#1311.

Technical details below:

The way the OS knows how to look up the data and present it is by
talking to the NSTextInput/NSTextInputClient. Previously MacVim
implemented NSTextInput partially, and didn't implement the critical
firstRectForCharacterRange:actualRange and characterIndexForPoint:
functions. First, in this change we change from NSTextInput to
NSTextInputClient (which is the newer non-deprecated version), and
implement those functions, which allows the OS to query the text
storage.

By default, the OS sends a quickLookWithEvent: call to us whenever the
lookup happens but for some odd reason this isn't automatic for Force
clicks, presumably because some apps want to handle Force clicks
manually (this is why some apps only work for three-finger taps but not
Force clicks for lookups). This isn't documented but I found references
in iTerm/Firefox, and basically we just need to manually handle it and
send off quickLookWithEvent: when handling Force clicks.

For implementing the NSTextInputClient properly, the main issue is
making sure that can work properly with input methods / marked texts,
which is the other primary purpose for this class (other than inputting
keys). For data lookups, I'm representing the grid as a row-major text
(with no newline/space in between) and expose that to the OS. This
already has some issue because it doesn't handle Vim vertical splits
well, as MacVim doesn't really have access to detailed Vim text buffers
easily (unless we do a lot of calls back-and-forth). This means wrapped
texts won't be looked up properly, which I think is ok. Also, the OS
APIs deal with UTF-8 indices, so we can't just convert row/column to raw
indices and have to do a lot of character length calculations
(especially for wide chars like CJK or emojis) to make sure the returned
ranges are consistent and valid. For marked texts though, this presents
a challenge because Vim doesn't really have a strong enough API to
communicate back-and-forth about the marked positions and whatnot (it
only let the GUI know where the current cursor is), and it's hard to
implement APIs like `markedRange` properly because some marked texts
could be hidden or wrapped (if you implement some of these functions
improperly Apple's input methods could start misbehaving especially when
you use arrow keys to navigate). In the end I kept the original
implementation for treating the marked texts as a range starting from 0,
*only* when we have marked text. Kind of a hack but this makes sure we
work both in marked text mode (i.e. when inputting texts) and when doing
lookups. For simplicity I made it so that you can't do data lookups when
in marked text mode now.

Input method:

This change also fixes a quirk in input method as a driveby change.
Previously the logic for calculating the rect for where the candidate
list was quite broken, but now it's calculated correctly using the
desired range and the current cursor position. This matters when say
using Japanese IM and using the left/right arrow to jump to different
sections of the text. If the desired range is in a wrapped line, the new
logic would attempt to pin it to the left-most column of where the
cursor is in the range.

Data detection:

Note that the default implementation is quite bare, and lacks a lot of
smart data detection. For example, if you put your mouse over a URL, it
won't properly select the whole URL, and addresses and dates for example
also won't get grouped together properly. This is because these require
additional implementation (e.g. using NSDataDetector) instead of coming
"for free", and will be handled later. In fact, Apple's WebKit and
NSTextView cheats by calling an internal API framework called "Reveal"
(which you can find out by intercepting NSTextView's calls and/or
looking at WebKit's source code) which is much more powerful and
supports looking up package tracking, airline info, and more, but it's
not available to third-party software (that's why Safari's lookup is so
much better than Chrome/Firefox's).

This isn't tested right now. Future task needs to add XCTest support to
properly test this as there are a lot of edge cases involved here.

Fix macvim-dev#1191
Part of Epic macvim-dev#1311, which contains other items to be implemented.
@ychin ychin force-pushed the text-input-client-support-lookup branch from 2928f77 to 83e925e Compare October 11, 2022 23:52
@ychin
Copy link
Member Author

ychin commented Oct 12, 2022

Input method fix where the candidate box is where it should be now when using left/right arrow keys to go back to previous sections.

image

@ychin ychin merged commit 80a7ca9 into macvim-dev:master Oct 12, 2022
@ychin ychin deleted the text-input-client-support-lookup branch October 12, 2022 04:28
ychin added a commit that referenced this pull request Feb 7, 2023
Updated to Vim 9.0.1276

Features
====================

Dictionary lookup
--------------------

You can now use Force Touch or Cmd-Ctrl-D to look up definitions of word
under the cursor (or selected text in visual mode). This will also
preview URLs, and support data types such as phone numbers and
addresses. #1312 #1313

This feature can also be invoked programmatically from VimScript (see
`:h macvim-lookup`). #1315

Tool bar / Touch Bar / menu icons
--------------------

You can now use SF Symbols for Tool bar and Touch Bar icons, including
using different symbol styles such as "palette" or "multicolor". Menu
items can now also use the `icon=` syntax to specify icons as well. See
`:help macvim-toolbar-icon` for details. #1329

The default tool bar also has updated icons to look similar to SF
Symbols used by newer macOS versions. #1214 by @sfsam

Window management actions
--------------------

There are new `macaction`'s for managing the MacVim window. The new
`zoomLeft`/`zoomRight` actions allow you to pin the window to the
left/right of the screen, and there are also new actions for interacting
with Stage Manager (requires macOS 13+). See `:h macvim-actions` for
details. #1330

Pre-release updates / Sparkle 2
--------------------

MacVim now supports pre-release software builds. It's sometimes hard for
us to release frequent updates due to the desire to pick a stable
upstream Vim version, needing to test the release on multiple OS
versions, making sure there aren't half-complete or buggy features, and
other reasons.

This new feature now allows us to push pre-release beta builds out in a
more frequent fashion, which could be useful if there are particular
features or fixes that you would like to try out before the next
official release. Pre-release builds will be released depending on bug
fixes and features instead of a fixed cadence. Do note that these
pre-release builds may not be as well-validated and may have half-baked
features.

If you are using the built-in auto-updater to update MacVim, you can
turn this on by going to Advanced settings pane, and enable "Enable
pre-release software updates".

This feature is only available for macOS 10.13 or above.

The auto-updater has also been updated from Sparkle 1.27.1 to 2.3.0 for
10.13+ builds. Legacy (10.9-10.12) builds are still using Sparkle 1.

See #1332.

New Vim features
--------------------

New `smoothscroll` option allows you to scroll through a long wrapped
line (using Ctrl-E or mouse wheel) without immediately jumping to the
next line. (v9.0.0640)

`splitscroll` option has been renamed `splitkeep`, with more flexibility
than before. (v9.0.0647)

Sound playback on macOS is now supported. You can use `has('sound')` to
check. See `help sound` for details. (v9.0.0694)

Terminals now support `:confirm` for `:q`, etc, which also means
MacVim's Cmd-W will work properly for terminal windows. (v9.0.0710)

Virtual text had numerous bugs fixed.

General
====================

Legacy build for 10.9 - 10.12
--------------------

Per a previous announcement (#1271), the default MacVim binary will now
require macOS 10.13 or above. Users of macOS 10.9 - 10.12 can use a
separate "legacy" build which will still be supported. The legacy binary
will still have the latest versions of Vim and be supported, but may not
have all the latest features (e.g. pre-release builds).

If you are using the auto-updater (Sparkle) to update MacVim, it should
"just work" and find the best version for you. If you are downloading
MacVim from the website, there is also a link to download the legacy
version marked for 10.9+ as well. If you download the normal binary
marked for 10.13+ from the website, it won't work on these older macOS
versions.

See #1331.

Fixes
====================

CoreText Renderer clipping and rendering bugs
--------------------

Unicode characters with multiple composing characters (e.g. "x⃗") will
now render correctly. #1172

Texts (e.g. Tibetan, Zalgo texts) that are taller than the line height
will no longer be clipped inappropriately. You can use a new setting
`MMRendererClipToRow` to re-enable clipping if the tall texts are
distracting. #995 / #1356

Tab crash
--------------------

Fixed a crash when opening new tabs that seems to only occur in macOS 13
Ventura. #1333

Other bugs
--------------------

- Fixed non-native full screen not working well with the notch on newer
  MacBook's when set to not show menu bar. You can also use
  `MMNonNativeFullScreenSafeAreaBehavior` to force MacVim to use the
  notch area as well if you don't mind some content being obscured. Note
  that the previous release also claimed it fixed this, but because the
  binary was built against an old macOS SDK (Big Sur), the fix did not
  work in the binary release. #1261
- Allow "Open untitled window: never" and "After last window closes:
  Quit MacVim" to be set together again. Added safeguards to make sure
  doing so won't immediately close the app. #1338
- Edit.Cut / Copy menu items will now be properly disabled when there
  isn't selected text. #1308
- Fixed potential `:emenu` crash when the menu is associated with an
  action in a non-valid mode. #1305
- Fixed bug where just bringing up the right-click (or the
  MacVim→Services) menu would somehow copy the selected texts to the
  system clipboard. #1300
- Fixed a Japanese input method bug where using left/right arrow to move
  to a different section of the input text would previously result in
  the candidate list not showing up at the correct position. #1312
- Fix non-CoreText renderer not handling text styles like strikethrough
  correctly (note: this renderer has been deprecated for a while and you
  should not use it). #1296
- This release uses an older sh/bash syntax file because the latest one
  in Vim has a bug. #1358

Misc
====================

New settings:

- "No drop shadows" (Appearance). #1301
- "Treat Ctrl-click as right-click" (Input) (#1326). This was previously
  configurable via command-line, but now also possible in the settings
  pane under the new "Input" category.

"About MacVim" now reports the version number in a clearer way with
clearly specified release number vs Vim version.

Known Issues
====================

Printing
--------------------

Printing using File→Print or `:hardcopy` is currently not working under
macOS 13 Ventura due to its removal of PostScript support in the Preview
app. This will be fixed in a later release. See the issue for
workarounds. #1347

Scripting
====================

- Scripting languages versions:
    - Perl is now built against 5.30, up from 5.18.
    - Ruby is now built against 3.2, up from 3.1.

Compatibility
====================

Requires macOS 10.9 or above. (10.9 - 10.12 requires downloading a
separate legacy build)

Script interfaces have compatibility with these versions:

- Lua 5.4
- Perl 5.30
- Python2 2.7
- Python3 3.10
- Ruby 3.2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for Look Up "<word>" popup menu
1 participant