Flashing in header files

[indianajohn]

I'm using YouCompleteMe together with the official Syntastic from scrooloose/syntastic. I have confirmed that this issue is present in the most recent version in Github.

I have a project set up so that all CPP files containing class declarations have a corresponding H file by the same name, but a different extension. Getting completion in the H files isn't particularly important for me,but when I don't provide the flags for that file, I get "can't find header..." errors. So I have set up YCM to provide the same flags for the header file as the CPP file.

However, now that I've provided the flags for the file, not only do I not get proper completions (I get cached completions, but if I type std::, for example, I don't get anything from STL), but I also get flickering every time I'm inserting text into the file.

I have narrowed it down to whenever VIM has a changed buffer stored for the .cpp file; if the other file is open but unaltered, no flickering occurs.

edit: I just checked the log files generated in /tmp, and nothing informative seems to be appearing. Just "initiating buffer for file blah.h", "initiating buffer for file blah.cpp" and "completion request for..." nothing else.

I've attached the contents of my .ycm_extra_conf.py file. Let me know if there's anything else I can provide to help you solve this problem.

Regards,
John

import os
import ycm_core
# Defaults, if no database can be found.
defaultsc = [
        'gcc',
        '-Wno-long-long',
        '-Wno-variadic-macros',
        '-pthread'
        '-std=c99',
        ]
defaultscpp = [
        'c++',
        '-Wno-long-long',
        '-Wno-variadic-macros',
        '-pthread'
        '-std=c++11',
        ]

# Things that must be included.
entered_flags = ['-fdelayed-template-parsing']

#
def DirectoryOfThisScript():
    return os.path.dirname( os.path.abspath( __file__ ) )

# Find all compilation databases in the build directory and load them.

def MakeRelativePathsInFlagsAbsolute( flags, working_directory ):
  if not working_directory:
    return list( flags )
  new_flags = []
  make_next_absolute = False
  path_flags = [ '-isystem', '-I', '-iquote', '--sysroot=' ]
  for flag in flags:
    new_flag = flag

    if make_next_absolute:
      make_next_absolute = False
      if not flag.startswith( '/' ):
        new_flag = os.path.join( working_directory, flag )

    for path_flag in path_flags:
      if flag == path_flag:
        make_next_absolute = True
        break

      if flag.startswith( path_flag ):
        path = flag[ len( path_flag ): ]
        new_flag = path_flag + os.path.join( working_directory, path )
        break

    if new_flag:
      new_flags.append( new_flag )
  return new_flags


def FlagsForFile( filename ):
    # Search through all parent directories for a directory named 'build'
    databases = []
    path_focus = os.path.dirname(filename)
    while len(path_focus) > 1:
        for f in os.listdir(path_focus):
            if f == 'build':
                compilation_database_folder = path_focus + "/" + f
                for r,d,f in os.walk(compilation_database_folder):
                    for files in f:
                        if files == 'compile_commands.json':
                            databases += [ycm_core.CompilationDatabase( r )]
        path_focus = os.path.dirname(os.path.dirname(path_focus))

    # Use a header's source file database for completion.
    filetype_flags = []
    if filename.endswith(".h"):
        for f in os.listdir(os.path.dirname(filename)):
            if filename.replace(".h",".cpp").find(f) != -1:
                filename = filename.replace(".h",".cpp")
                break
            if filename.replace(".h",".c").find(f) != -1:
                filename = filename.replace(".h",".c")
                break
    elif filename.endswith(".hpp"):
        for f in os.listdir(os.path.dirname(filename)):
            if filename.replace(".hpp",".cpp").find(f) != -1:
                filename = filename.replace(".hpp",".cpp")
                break

    # Get the compile commands
    final_flags = []
    # If possible, from the database.
    if len(databases) > 0:
        for database in databases:
            compilation_info = database.GetCompilationInfoForFile( filename )
            fromfile_flags = MakeRelativePathsInFlagsAbsolute(
            compilation_info.compiler_flags_,
            compilation_info.compiler_working_dir_ )
            final_flags += fromfile_flags

    # If not, set some sane defaults.
    else:
        relative_to = DirectoryOfThisScript()
        final_flags += MakeRelativePathsInFlagsAbsolute( flags, relative_to )
    if not final_flags:
        if filename.endswith(".c"):
            final_flags = defaultsc
        elif filename.endswith(".cpp"):
            final_flags = defaultscpp

    # This allows header files to be parsed according to their parent source
    final_flags = filetype_flags + final_flags

    # For things that must be included regardless:
    final_flags += entered_flags
    return {
        'flags': final_flags,
        'do_cache': True
    }

[Valloric]

Try to make a small, minimal test case & test procedure that reproduces the issue. Until then, I'm assuming this is related to your configuration (seems most likely) and not a YCM bug.

BTW remove the paths to compiler binaries in your flags.

[indianajohn]

I have narrowed this down further. It actually doesn't seem to have anything to do with .h/.cpp files. Two conditions are needed:
-Both YouCompleteMe and Syntastic are installed.
-"set hidden" is active in .vimrc
-gvim (not plain vim) is used.

I've produced this in both the version of Vim in the the Debian Testing repositories and a recent version compiled from source.

To reproduce:
-Install YouCompleteMe/Syntastic (Nothing else)
-Put "set hidden" in .vimrc
-Open one cpp file
-Ensure that YCM is active somehow, by feeding it proper flags through .ycm_extra_conf.py.
-Make a change to that buffer without saving it.
-Open another file.
-Start typing. You should notice a flickering.

Or, just:

mkdir ~/dev; cd ~/dev
git clone http://github.com/indianajohn/ycm-bug/
cd ycm-bug
./install.sh

(wait)

gvim hello-world.cpp &

(insert some text without saving)

:o goodbye-cruel-world.cpp

(enter insert mode and start typing).

[Valloric]

1. What's the output of vim --version?
2. What OS are you using?

[indianajohn]

1. Version

$ vim --version
VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Aug 17 2013 13:47:09)
Included patches: 1-5
Compiled by john@john-desktop
Huge version with GTK2 GUI.  Features included (+) or not (-):
+arabic          +file_in_path    +mouse_sgr       +tag_binary
+autocmd         +find_in_path    -mouse_sysmouse  +tag_old_static
+balloon_eval    +float           +mouse_urxvt     -tag_any_white
+browse          +folding         +mouse_xterm     -tcl
++builtin_terms  -footer          +multi_byte      +terminfo
+byte_offset     +fork()          +multi_lang      +termresponse
+cindent         +gettext         -mzscheme        +textobjects
+clientserver    -hangul_input    +netbeans_intg   +title
+clipboard       +iconv           +path_extra      +toolbar
+cmdline_compl   +insert_expand   -perl            +user_commands
+cmdline_hist    +jumplist        +persistent_undo +vertsplit
+cmdline_info    +keymap          +postscript      +virtualedit
+comments        +langmap         +printer         +visual
+conceal         +libcall         +profile         +visualextra
+cryptv          +linebreak       +python          +viminfo
+cscope          +lispindent      -python3         +vreplace
+cursorbind      +listcmds        +quickfix        +wildignore
+cursorshape     +localmap        +reltime         +wildmenu
+dialog_con_gui  -lua             +rightleft       +windows
+diff            +menu            +ruby            +writebackup
+digraphs        +mksession       +scrollbind      +X11
+dnd             +modify_fname    +signs           -xfontset
-ebcdic          +mouse           +smartindent     +xim
+emacs_tags      +mouseshape      -sniff           +xsmp_interact
+eval            +mouse_dec       +startuptime     +xterm_clipboard
+ex_extra        -mouse_gpm       +statusline      -xterm_save
+extra_search    -mouse_jsbterm   -sun_workshop    
+farsi           +mouse_netterm   +syntax          
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "$VIM/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/usr/share/vim"
 f-b for $VIMRUNTIME: "/usr/share/vim/vim74"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK  -pthread -I/usr/include/gtk-2.0 -I/usr/lib/x86_64-linux-gnu/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/pango-1.0 -I/usr/include/gio-unix-2.0/ -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12   -I/usr/local/include  -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1      
Linking: gcc   -L. -Wl,-Bsymbolic-functions -Wl,-z,relro -rdynamic -Wl,-export-dynamic  -L/usr/local/lib -Wl,--as-needed -o vim   -lgtk-x11-2.0 -lgdk-x11-2.0 -latk-1.0 -lgio-2.0 -lpangoft2-1.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lcairo -lpango-1.0 -lfreetype -lfontconfig -lgobject-2.0 -lglib-2.0   -lSM -lICE -lXpm -lXt -lX11 -lXdmcp -lSM -lICE  -lm -ltinfo -lnsl  -lselinux  -ldl    -L/usr/lib/python2.7/config -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions   -lruby1.8 -lpthread -lrt -ldl -lcrypt -lm  -L/usr/lib

1. This bug is reproducible both in Ubuntu 12.04 LTS x64 with the version of vim compiled from source above, and in Debian Testing with the version of vim in apt-get (vim-gnome? gnome-vim?). Which is interesting, because one uses gtk2, and the other uses gnome.

[netsmertia]

I have same flickering problem with same version of vim. Firstly I thought it is due to problem in my vimrc but now it seems a YCM bug. It is confirmed if you are editing two unsaved buffer window will flicker. And if i disable YCM, no flicker.
OS ubuntu13.04 x64 vim 7.4 Gui vim-gtk

[Svalorzen]

Same problem here. My current workaround for when I see flicker is :wall. However, differently from indianajohn, in my .vimrc I have explicitly set nohidden, so probably that does not influence the problem.
Vim version:

VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Oct 13 2013 16:49:55)
Included patches: 1-52
Compiled by svalorzen@Luchino
Huge version with GTK2 GUI.  Features included (+) or not (-):
+acl             +farsi           +mouse_netterm   +syntax
+arabic          +file_in_path    +mouse_sgr       +tag_binary
+autocmd         +find_in_path    -mouse_sysmouse  +tag_old_static
+balloon_eval    +float           +mouse_urxvt     -tag_any_white
+browse          +folding         +mouse_xterm     -tcl
++builtin_terms  -footer          +multi_byte      +terminfo
+byte_offset     +fork()          +multi_lang      +termresponse
+cindent         +gettext         -mzscheme        +textobjects
+clientserver    -hangul_input    +netbeans_intg   +title
+clipboard       +iconv           +path_extra      +toolbar
+cmdline_compl   +insert_expand   -perl            +user_commands
+cmdline_hist    +jumplist        +persistent_undo +vertsplit
+cmdline_info    +keymap          +postscript      +virtualedit
+comments        +langmap         +printer         +visual
+conceal         +libcall         +profile         +visualextra
+cryptv          +linebreak       +python          +viminfo
+cscope          +lispindent      -python3         +vreplace
+cursorbind      +listcmds        +quickfix        +wildignore
+cursorshape     +localmap        +reltime         +wildmenu
+dialog_con_gui  -lua             +rightleft       +windows
+diff            +menu            +ruby            +writebackup
+digraphs        +mksession       +scrollbind      +X11
+dnd             +modify_fname    +signs           -xfontset
-ebcdic          +mouse           +smartindent     +xim
+emacs_tags      +mouseshape      -sniff           +xsmp_interact
+eval            +mouse_dec       +startuptime     +xterm_clipboard
+ex_extra        -mouse_gpm       +statusline      -xterm_save
+extra_search    -mouse_jsbterm   -sun_workshop    +xpm
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "$VIM/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/usr/share/vim"
 f-b for $VIMRUNTIME: "/usr/share/vim/vim74"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK  -pthread -I/usr/include/gtk-2.0 -I/usr/lib/x86_64-linux-gnu/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/pango-1.0 -I/usr/include/gio-unix-2.0/ -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12 -I/usr/include/harfbuzz   -I/usr/local/include  -g -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1      
Linking: gcc   -L. -Wl,-Bsymbolic-functions -Wl,-z,relro -rdynamic -Wl,-export-dynamic  -L/usr/local/lib -Wl,--as-needed -o vim   -lgtk-x11-2.0 -lgdk-x11-2.0 -latk-1.0 -lgio-2.0 -lpangoft2-1.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lcairo -lpango-1.0 -lfreetype -lfontconfig -lgobject-2.0 -lglib-2.0   -lSM -lICE -lXpm -lXt -lX11 -lXdmcp -lSM -lICE  -lm -ltinfo -lnsl  -lselinux  -ldl    -L/usr/lib/python2.7-config -lpython2.7   -lruby-1.9.1 -lpthread -lrt -ldl -lcrypt -lm  -L/usr/lib   

On Lubuntu 13.10.

[Valloric]

This might need to be investigated further.

[meh]

I think the flickering I'm seeing is related to this, except it's not that consistent, using vim from the hg repository and latest YCM and Syntastic.

[Valloric]

I can confirm this issue. I think it might be a Syntastic bug, but I'll have to explore further.

[meh]

@Valloric since syntastic won't fix the bug, can we at least get a version check before setting the lazy redraw?

[Valloric]

@meh That's what I've been thinking of, but I'm not sure which version of Vim has the bugfix that the lazy redraw fixes.

[meh]

@Valloric how about a compatibility flag to disable the workarounds for old versions?

[Valloric]

@meh Actually what I'll do is just remove the use of the lazy redraw entirely. I'd rather have the super-rare Vim crash in old versions of Vim than have the incredibly annoying flashing which occurs much more often.

So the workaround for #593 is going to be reverted. @oblitum Do you know which version of Vim has your bugfix? I'd like to mention it in the FAQ.

[Svalorzen]

I just updated YCM but I have to say that for me the bug is still there. As long as any other buffer is open and unsaved, any edit to the file causes the screen to flicker. I have to say that it does not happen if I simply move the cursor (be it in normal mode or insert mode).

It flickers once always when exiting insert mode for normal mode (but not when I enter insert mode from normal mode). And happens every time I add or remove a character in insert mode. Visual mode seems to be unaffected.

It flickers even if syntastic is toggled off.

[Valloric]

@Svalorzen It appears you are correct. Reopening.

[Valloric]

I finally tracked this down and worked around the Vim bug. Here's what I wrote in the commit message:

On every key press, we end up calling GetUnsavedAndCurrentBufferData(), which
calls GetBufferOption( buffer_object, 'ft' ). If the buffer_object represents a
hidden buffer, Vim would flicker.

This would happen because we'd call "buffer_object.options[ 'ft' ]" in recent
versions of Vim, and that line of code causes Vim to flicker. I don't know why.
We're extracting the 'ft' value without going through buffer_object.options, and
that works just fine.

@oblitum You know the Vim codebase pretty good, could you take a look at the root cause of this? I've worked around the problem so it's not a pressing issue, but executing buffer_object.options[ 'ft' ] for a hidden buffer should not cause the screen to flicker. It's obviously a bug in Vim.

[oblitum]

@Valloric sorry, I entirely missed the discussion that was happening here. Answering the month older question, the version with the bugfix for #593 (patched file: popupmnu.c) is:

revision:      5444:d563839a9be0
tag:           v7-4-072
user:          Bram Moolenaar <bram@vim.org>
date:          Wed Nov 06 04:04:33 2013 +0100
description:   updated for version 7.4.072

(Bram ended up applying a somewhat different patch which also worked)

---

About the flickering problem, on my machine it's somewhat quite subtle so I didn't notice there was such an issue, but anyway, debugging I found the problem lies with the lines 8969 and 8971 of the option.c file. Commenting out those lines, the flickering vanishes. That calls end up doing a lot of buffer/window operations when there's a hidden edited buffer, I think unecessarily. They stand as guarding calls around the 8970 that does the useful (and what should have been simple) job of getting a buffer property. It must be fixed, when I get the time I'll debug a bit more and post an issue or patch at the mailing list.

[oblitum]

About this being a GVIM specific problem, it's not since I've noticed the subtle flickering on terminal (and debugged vim there).

[Valloric]

@oblitum Thanks for looking into this! You're a gem, dude. I'm sure Bram would love to see a patch for this.

Like I said, this is not critical at all since YCM has a good workaround. I'd just like to prevent a situation where someone else hits the same bug; it took me the better part of a month to track down the root cause of this nonsense.