Pasting is slow when using bracketed-paste-magic

[aaronjensen]

Because every character pasted attempts to autosuggest, pasting something is slow.

Ideally this plugin would support bracketed paste mode out of the box (or the completions would be async as proposed in #134

[ericfreese]

Will look into what I can do to speed the paste up when using bracketed-paste-magic. In the mean time, the workaround from #102 (comment) will work to speed up pasting:

zstyle ':bracketed-paste-magic' active-widgets '.self-*'

Let me know if you'd like me to split this into two issues.

Yeah, can you create a separate issue for interaction with safe-paste? We can just keep this particular issue about slow pasting with bracketed-paste-magic.

[ericfreese]

The best solution I can think of right now would be to add a hook to paste-init that temporarily disables suggestion-fetching and a hook to paste-finish that re-enables it. Using the active-widgets zstyle to detect if bracketed-paste-magic is enabled.

Not the most elegant solution though... I'm gonna sleep on this one a bit, and see if I can come up with something a little nicer.

[aaronjensen]

I can't say I noticed much if any change w/ zstyle ':bracketed-paste-magic' active-widgets '.self-*', it's still just as slow to paste as far as I can tell.

[ericfreese]

Logging another possible solution here:

bindkey '^[[200~' to a function that disables suggestions and '^[[201~' to a function that re-enables them.

I think I like this better than the solution above. Would be a little more general than detecting and dealing with bracketed-paste-magic specifically.

[aaronjensen]

do bindkeys stack? Would that not interfere with bracketed-paste-magic? If not, seems simple enough

[lbolla]

This is affecting me, too. Using zsh-autosuggestions.zsh v0.3.1. Workaround didn't work (I am not using oh-my-zsh or other plugins, btw).

[ericfreese]

@lbolla

I am not using oh-my-zsh or other plugins, btw

Are you using bracketed-paste-magic?

Do you see the issue without any other config? Try something like this:

% zsh -f
%% source zsh-autosuggestions.zsh
%% [paste something long]

[lbolla]

@ericfreese I tried what you suggested, but pasting is still slow. I can try to create an animation to show how slow, if that helps. I tried using different terminals (Gnome Terminal and xterm), just in case: same result.

[ericfreese]

@aaronjensen Please try the fixes/slow_bracketed_paste_magic branch, and let me know if it solves your issues.

[ericfreese]

zprof output of pasting long command from zsh-users/zsh-syntax-highlighting#295 (comment)

• Before
• After

[aaronjensen]

@ericfreese sorry, still slow :/ w/o zsh-autosuggestions and my config, paste is near instant. with it, it's slow. Trying from zsh -f it's faster, but still not as fast as my full config w/o zsh-autosuggestions.

[jasontibbitts]

I have basically the same issue, but for me it depends on the version of zsh in use. On my Fedora 23 machines with zsh 5.2, pasting is always instantaneous. But when I ssh to a RHEL7 machines with zsh 5.0.2, it's slow. I suspect that 5.0.2 is simply to old to support bracketed paste, but I'm not sure.

[aaronjensen]

I just tried upgrading to zsh 5.2 and didn't see a performance improvement, even w/ https://github.com/zsh-users/zsh-autosuggestions/tree/fixes/slow_bracketed_paste_magic

[derimagia]

If you have a high enough version of zsh (5.2), I recommend using https://github.com/zsh-users/zsh/blob/master/Functions/Zle/bracketed-paste-url-magic (See http://www.zsh.org/mla/workers/2015/msg02610.html) as an alternative.

[aaronjensen]

@derimagia thanks, but that's not the problem here.

@ericfreese Pasting with zsh-autosuggstions is still slow even with disabling bracketed-paste-magic, 9fb9675, zsh 5.2, and/or

zstyle ':bracketed-paste-magic' active-widgets '.self-*'

Should one of those have worked for me, or are you still working/thinking on this one? Thanks!

[lbolla]

Like others, I am experiencing slow paste even if I am not using bracketed-paste-magic.
I digged a bit into zsh-autosuggest code and found that the slowness is mainly due to too many levels of indirection when selecting the autosuggest function.
As a test, I tried to manually inline the "default" strategy in _zsh_autosuggest_modify:

diff --git a/zsh-autosuggestions.zsh b/zsh-autosuggestions.zsh
index 4e3c62e..707f745 100644
--- a/zsh-autosuggestions.zsh
+++ b/zsh-autosuggestions.zsh
@@ -243,7 +243,17 @@ _zsh_autosuggest_modify() {
    # Get a new suggestion if the buffer is not empty after modification
    local suggestion
    if [ $#BUFFER -gt 0 ]; then
-       suggestion="$(_zsh_autosuggest_suggestion "$BUFFER")"
+       # suggestion="$(_zsh_autosuggest_suggestion "$BUFFER")"
+
+            # TODO inline _zsh_autosuggest_strategy_default
+                local prefix="$BUFFER"
+
+                # Get the keys of the history items that match
+                local -a histkeys
+                histkeys=(${(k)history[(r)$prefix*]})
+
+                # Echo the value of the first key
+                suggestion="${history[$histkeys[1]]}"
    fi

    # Add the suggestion to the POSTDISPLAY

(The code is in a branch in my own fork: lbolla@dac5b7f)

The result is an overall speed up of autosuggestion in all cases, especially when pasting (with mouse middle click) clipboard data.

[lbolla]

Another effective workaround is to disable autosuggestion if $BUFFER is too large.
This effectively means that the autosuggest function won't be called many times (e.g. if you are pasting a very large chunk of text, it'll be called just a few times).

diff --git a/zsh-autosuggestions.zsh b/zsh-autosuggestions.zsh
index 4e3c62e..7a74ab0 100644
--- a/zsh-autosuggestions.zsh
+++ b/zsh-autosuggestions.zsh
@@ -242,7 +242,7 @@ _zsh_autosuggest_modify() {

        # Get a new suggestion if the buffer is not empty after modification
        local suggestion
-       if [ $#BUFFER -gt 0 ]; then
+       if [ $#BUFFER -gt 0 -a $#BUFFER -lt 20 ]; then
                suggestion="$(_zsh_autosuggest_suggestion "$BUFFER")"
        fi

See lbolla@9572ccb

[tonybaroneee]

@lbolla thanks for that!

[balta2ar]

@lbolla Could you please send these two changes as pull requests so that it's easies for @ericfreese to merge them if he wants to?
As a side note, I'm really looking forward to seeing async implemented natively. I'm currently sitting on that abandoned zsh-async branch and it really saves my day. I can't use zas with it :(

[lbolla]

Issued PR #177

[prasanthkrishnan]

Same issue here, without zsh-autosuggestions my config paste instantly.
w/o bracketed-paste-magic
OSX
Zsh 5.2

Surprisingly setting ZSH_AUTOSUGGEST_BUFFER_MAX_SIZE didnt help :( .

[lbolla]

@prasanthkrishnan how large is the text you tried to paste? What did you use for ZSH_AUTOSUGGEST_BUFFER_MAX_SIZE? Also, consider that you need to set ZSH_AUTOSUGGEST_BUFFER_MAX_SIZE before sourcing zsh-autosuggest.

[aaronjensen]

ZSH_AUTOSUGGEST_BUFFER_MAX_SIZE worked for me. It appears that I needed to set that after sourcing zsh-autosuggest, otherwise zsh-autosuggest cleared it out.

[prasanthkrishnan]

In zsh -f pasting is fine, even without setting setting ZSH_AUTOSUGGEST_BUFFER_MAX_SIZE
:(
I dont use oh-my-zsh or presto. I do use zsh-syntax-highlighting, zsh-navigations-tools, zsh-completions.

[aaronjensen]

@prasanthkrishnan maybe silly question, but are you on the latest master fedc22e for zsh-autosuggestions?

[prasanthkrishnan]

@aaronjensen You caught me, I was using 0.3.2. Switched to 0.3.3. ZSH_AUTOSUGGEST_BUFFER_MAX_SIZE works now.
It pastes char by char till max_size and completes right after the limit :P
Thanks a lot for the help.

[aaronjensen]

Awesome. Glad that worked. I'm going to close this because it seems fixed. Thanks @lbolla

[aaronjensen]

Oh one other thing, it might be nice to have a sensible default for this.

[balta2ar]

@aaronjensen I had a similar issue and didn't really pay attention until today I stumbled upon a working solution. I'm using zsh 5.3.1 with bracketed-paste-magic enabled, zsh-autosuggestions are from today's develop branch with async changes (but I don't think the latter is important). Putting code from this message (http://zsh-users.zsh.narkive.com/mhgulrfD/bracketed-paste-mode-in-xterm-and-urxvt#post1) helped immensely. Now paste is instant. I want to stress that I'm not using ZSH_AUTOSUGGEST_BUFFER_MAX_SIZE. Hope that helps.

Here is the suggested code just in case the link dies:

# create a new keymap to use while pasting
bindkey -N paste
# make everything in this keymap call our custom widget
bindkey -R -M paste "^@"-"\M-^?" paste-insert
# these are the codes sent around the pasted text in bracketed
# paste mode.
# do the first one with both -M viins and -M vicmd in vi mode
bindkey '^[[200~' _start_paste
bindkey -M paste '^[[201~' _end_paste
# insert newlines rather than carriage returns when pasting newlines
bindkey -M paste -s '^M' '^J'

zle -N _start_paste
zle -N _end_paste
zle -N paste-insert _paste_insert

# switch the active keymap to paste mode
function _start_paste() {
    bindkey -A paste main
}

# go back to our normal keymap, and insert all the pasted text in the
# command line. this has the nice effect of making the whole paste be
# a single undo/redo event.
function _end_paste() {
    #use bindkey -v here with vi mode probably. maybe you want to track
    #if you were in ins or cmd mode and restore the right one.
    bindkey -e
    LBUFFER+=$_paste_content
    unset _paste_content
}

function _paste_insert() {
    _paste_content+=$KEYS
}

function _zle_line_init() {
    # Tell terminal to send escape codes around pastes.
    [[ $TERM == rxvt-unicode || $TERM == xterm ]] && printf '\e[?2004h'
}

function _zle_line_finish() {
    # Tell it to stop when we leave zle, so pasting in other programs
    # doesn't get the ^[[200~ codes around the pasted text.
    [[ $TERM == rxvt-unicode || $TERM == xterm ]] && printf '\e[?2004l'
}

Rest of the message

Alternatively, you can also do stuff to the text before inserting it, I have this additional stuff which lets me toggle a mode where all the pasted text is automatically quoted and a space is appended, which is useful when pasting (some) urls with ? and & and what have you.

function _end_paste() {
bindkey -e
if [[ $_SPACE_AFTER_PASTE_QUOTE = 1 ]]; then
LBUFFER+=${(q)_paste_content}' '
else
LBUFFER+=$_paste_content
fi
unset _paste_content
}

function _spaceafterpastequote() {
if [[ $_SPACE_AFTER_PASTE_QUOTE = 1 ]]; then
_SPACE_AFTER_PASTE_QUOTE=0
zle -M "Not inserting a space after pastes, not quoting"
else
_SPACE_AFTER_PASTE_QUOTE=1
zle -M "Inserting a space after pastes and quoting"
fi
}
zle -N _spaceafterpastequote
# this is a custom wrapper that uses zkbd stuff, just use regular bindkey.
zbindkey Control-Insert _spaceafterpastequote

[balta2ar]

@ericfreese I wonder why I didn't have have any effect on pasting speed with turned on bracketed paste mode until I yesterday added the code from my comment above. Your understanding of zsh is way better, do you, by any chance, know why that might be the case?

Sorry for scattering discussions around, but do you think this same technique could be used to help with #136? I've tried changing the code above to this hoping that cd and \n will enable and disable paste mode correspondingly:

bindkey ' cd ' _start_paste
bindkey -M paste "\n" _end_paste

but it only made mc hang upon the start. Basically I thought of a way to disable ANY zsh hooks altogether, just as in bracketed paste mode, before mc sends its long cd path and enable hooks after the path has been sent. mc sends commands likes this: cd "printf "%b" '\0057sbin'". They start with <space>cd<space> and end with \n. Is there any way to disable all hooks based on this pattern? If I could do this, I think this would have solved all my performance issues with cd in mc.

[aaronjensen]

@balta2ar Thanks, though it looks like that suggestion, if I understand it correctly, is effectively disabling and replacing bracketed paste magic.