Vim colorscheme leaves a wake of destruction when switching away

[tpope]

Easiest way to demonstrate:

:e ~/.vimrc
:colorscheme solarized
:colorscheme default

Note that the highlighting for vimCommand and other highlight groups is gone.

I haven't figured out the precise trigger yet (I'll probably take a closer look this weekend), but I can tell you that it never seems to happen if you :hi link rather than directly override a group's colors.

[altercation]

I'll look into it as well. The only autocommand I use on colorscheme
switching is to unload the Solarized menu. I have some ideas, will
explore.

[tpope]

I don't think it's the autocommand. I think it has to do with the fact vimCommand is hi def linked in syntax/vim.vim, and once you've assigned colors, and Vim has cleared them out, it will never assign them again.

[altercation]

What's a better test for this? Unless I'm wrong vimCommand just isn't
defined by default at all, so it's not going to get a new value. Or am
I missing the point here?

[tpope]

From syntax/vim.vim:

hi def link vimCommand  Statement

That means "unless vimCommand is already defined, link it to Statement" (see :help :hi-default). Solarized declares it to be yellow. When you switch away from Solarized, Vim resets all the highlighting. But when syntax/vim.vim is reapplied, vimCommand is already "tainted", so the highlight default no longer applies. Ergo, it's left cleared.

At least, that's my best understanding of it. Clearly it's an incomplete explanation as other groups appear fine.

[altercation]

Not really a Solarized issue then, no? I could make a universal syntax loader in an ftdetect or just a BufNewFile/BufRead to preempt syntax detection and try to reset the syntax easily enough.

[tpope]

Well, it's only a Solarized issue in that Solarized is doing something Vim doesn't seem to support. The ability to override highlight group linkings was intended to be user specific rather than colorscheme specific. So, it's not surprising that there's some friction here.

I do have hope, though, as rubyDefine, a seemingly identical case, works fine.

[altercation]

Tim, can you point me to the docs on this being bad form in a colorscheme? I buy what your saying but just didn't run across it, or didn't pay enough attention when I did. I'm sure there is other stuff I've been out of bounds on as well and really would like to ensure that this is a good example of form, not just function, by complying with guidelines as I can.

I see a couple options:

1. Don't link at all. This seems like a poor choice as linking is a more efficient method in my mind (efficient in terms of management of the colorscheme), rather than making identical highlight statements, but if it solves this and is best practice, so be it.
2. Create an autocmd in Solarized that simply does a full clear and reset on the syntax when a new colorscheme is applied. More of a kludge and I have tried to avoid piling on autocmds, but it's there as an option. I could even unload the autocommand in the autocommand just to keep things neat.

Thoughts? Other directions?

[tpope]

The Vim help isn't too explicit about this, but if you read :help :hi-default, you'll see the example they give is overriding a highlight group in one's vimrc. Putting it there implies it will last through a colorscheme change.

Switching away from linking isn't going to solve it. In fact vimCommand isn't even linked. When I switch it to being linked, it changes it from failing to highlight when switching colorschemes to highlighting incorrectly when switching color schemes.

I don't think a full clear is going to cut it, as that already happens when switching colorschemes. Vim provides :hi clear but doesn't seem to provide a way to delete a group entirely (so the default can take hold again). Once I break things, I can't find a way to restore them, short of a :hi link vimCommand Statement. Well, I guess you could manually link each and every one back.

The only other solution I can think of right now is to separate out the basic color definitions from all the filetype specific stuff, so that people that want to switch colorschemes can opt out of the latter. I hope to top that.

[altercation]

Tim, I don't know if this is the right approach, but if I run the following starting in solarized (in a new buffer for simplicity):

:setf vim | hi vimCommand | colorscheme default | hi vimCommand

(returns)

vimCommand     xxx guifg=#b58900
vimCommand     xxx cleared

I think we can actually reset more or less fully by setting the filetype again (in the same buffer):

:setf vim

Which results in:

:hi vimCommand
vimCommand     xxx links to Statement 

I could automate this into an autocmd. I might be missing something here, so don't let me head off into the bush on this.

[altercation]

I have this provisionally, seems to do the trick:

autocmd ColorScheme * if g:colors_name != "solarized" | call s:SolarizedUnload() | else | call SolarizedMenu() | endif

function! s:SolarizedUnload()
    silent! aunmenu Solarized
    silent! exe 'set filetype='.&filetype
endfunction

[tpope]

Bizarrely, this seems to work the first time, but if I cycle colorschemes again, it seems to break.

[altercation]

Dammit, you're right.

[lusis]

I was actually going to open a ticket on this earlier. For the longest time I was attempting to test solarized and assumed it was just broken. Reason was as @tpope said I was attempting to load it by :color solarized.

[altercation]

@lusis, calling :color solarized should still work (at least as long as you aren't toggling between schemes). Are you seeing a specific issue calling solarized from a different scheme?

[lusis]

I was originally using a twilight colorscheme port. When ever I cycled to solarized, the only thing I got was a single font color and the background color change. When I forced solarized for a given filetype, it worked great until I started cycling colorschemes again. I don't do it that often but I do use different colorschemes for python vs ruby so occasionally gvim cycles on me automatically when I switch language contexts. I can probably duplicate it again fairly easily.

[tpope]

@lusis, if you are changing colorschemes on a per filetype basis, I can tell you you're definitely twisting Vim in ways it was never meant to be twisted (and thus it's not really surprising that things might break, even if the particular reason isn't immediately obvious). What happens if you have a window split with a Python and Ruby file open at the same time?

[lusis]

@tpope I totally agree. The only time it bites me in the ass is when I open a JSON file while working on Ruby. I never find myself working on Python and Ruby in the same "session". It's mainly there to help with the mental context switching when I swap projects.

As to the original issue I saw, I wasn't actually doing any filetype switching. I was loading vim with my original colorscheme. I typed :color solarized and saw nothing like the pictures. Everytime I updated my submodules, I'd give it another shot. It was only until I thought about starting vim with solarized that it worked. As you found though, the same effect is easily duplicatable by simply cycling through colorschemes and coming back to solarized.

[mattsacks]

I've narrowed it down to something that has to do with hi def link instead of using hi link. I don't really know why this would only apply to Solarized, but here's some steps you can follow along with to see the behavior:

for reference, I'm using my fork of vim-javascript but it also applies to the origin as well

1. Edit a javascript file in Vim with solarized set as colors_name
2. Change colorscheme to something else
3. Change colorscheme back to solarized
4. Notice how some things aren't properly highlighted like hi javaScriptLabel gets cleared
5. Go into syntax/javascript.vim (with fugitive it's just :Vvsplit syn/javasc) and change all the HiLink statements at the bottom to hi link (since the command HiLink uses hi def link if supported)
6. Save the syntax file and go back to the javascript one
7. Edit the current file with :e %
8. Highlighting should be back again

That's as far as I've gotten. Not a good solution since you'll have to change it back to HiLink for custom highlighting outside of the syntax file, but hope it helps figure out what's wrong.

[altercation]

That's definitely good intel and a solid repro case. Thanks.

[xolox]

Hi all,

Sorry to barge in here a year after the last comment and suggest to close an issue on a project I'm not involved in (although I am a happy user :-) however I believe that: 1. this is caused by a bug in Vim and 2. I've found a fairly elegant workaround. So I suppose this issue can be closed?

Several years ago I wrote a color scheme switcher plug-in for Vim but never published it. Shortly after creating my plug-in I noticed the behavior described in this issue and became annoyed by it, convinced that it was a bug in Vim. I even went so far as to try and fix the issue inside the Vim C code but that was a bit too much for me :-]. Back then I had never heard of Solarized; I encountered the problem with a couple of other color schemes (including my own).

Today I decided to publish the color scheme switcher plug-in (see xolox/vim-colorscheme-switcher) and before I published the plug-in I decided to take another stab at fixing the problem described here. I tried all sorts of nasty hacks but ended up with a fairly elegant workaround that doesn't seem too fragile. The readme / homepage contains an explanation of how it works (I'm quoting it below for your convenience).

---

The way Vim color schemes are written isn’t really compatible with the idea of quickly switching between lots of color schemes. In my opinion this is an ugly implementation detail of how Vim works internally, in other words I think it’s a bug that should be fixed… Here are some references that explain the problem in some detail:

• Vim colorscheme leaves a wake of destruction when switching away
• gVim: remove syntax highlighting groups

Since this behavior hinders cycling through color schemes, the color scheme switcher plug-in includes a workaround that should hide the problem:

1. At startup a dictionary is created which will be used to remember links between highlighting groups.
2. Before and after loading a color scheme, the color scheme switcher plug-in runs the :highlight command without any arguments to find links between highlighting groups. Each link that is found is added to the dictionary. Existing entries are updated. This is done by calling xolox#colorscheme_switcher#find_links().
3. After loading a color scheme, the color scheme switcher plug-in runs the :highlight command without any arguments to find highlighting groups in the state ‘cleared’. For each of these groups, if they were previously linked, the link is restored. This is done by calling xolox#colorscheme_switcher#restore_links().

Probably this solution is still not perfect, but it’s a lot better than the behavior out of the box: Before I implemented the steps above, when I would cycle through my color schemes, Vim would eventually end up with black text on a white background and nothing else! With the steps above, I can cycle as many times as I want and all of the color schemes I’ve checked so far look fine.

---

I would love feedback on the (technique used by the) color scheme switcher; I hope it works as well for you as it does for me. I tested it in Vim 7.3 on Linux (GTK) and in MacVim 7.3.

[auwsmit]

Sorry for resurrecting this thread, but this "bug" is still a problem in Vim 8.

As xolox details in the post above this one, he found a workaround that fixes the issue. However, he only implemented this fix within his plugin, and the plugin doesn't allow for switching to a specific colorscheme by name. I took the liberty of extracting the relevant functions and making a custom :MyColorscheme command which doesn't have this issue like :colorscheme does.

If anyone stumbles across this problem in the future and is looking for a quick fix, simply add this to your vimrc, use :MyColorscheme to switch colorschemes, and voila!:

if !exists('s:known_links')
  let s:known_links = {}
endif

function! s:Find_links() " {{{1
  " Find and remember links between highlighting groups.
  redir => listing
  try
    silent highlight
  finally
    redir END
  endtry
  for line in split(listing, "\n")
    let tokens = split(line)
    " We're looking for lines like "String xxx links to Constant" in the
    " output of the :highlight command.
    if len(tokens) == 5 && tokens[1] == 'xxx' && tokens[2] == 'links' && tokens[3] == 'to'
      let fromgroup = tokens[0]
      let togroup = tokens[4]
      let s:known_links[fromgroup] = togroup
    endif
  endfor
endfunction

function! s:Restore_links() " {{{1
  " Restore broken links between highlighting groups.
  redir => listing
  try
    silent highlight
  finally
    redir END
  endtry
  let num_restored = 0
  for line in split(listing, "\n")
    let tokens = split(line)
    " We're looking for lines like "String xxx cleared" in the
    " output of the :highlight command.
    if len(tokens) == 3 && tokens[1] == 'xxx' && tokens[2] == 'cleared'
      let fromgroup = tokens[0]
      let togroup = get(s:known_links, fromgroup, '')
      if !empty(togroup)
        execute 'hi link' fromgroup togroup
        let num_restored += 1
      endif
    endif
  endfor
endfunction

function! s:AccurateColorscheme(colo_name)
  call <SID>Find_links()
  exec "colorscheme " a:colo_name
  call <SID>Restore_links()
endfunction

command! -nargs=1 -complete=color MyColorscheme call <SID>AccurateColorscheme(<q-args>)

[krlawrence]

Thanks for posting this. I was having similar issues with the Afterglow color scheme.

Cheers
Kelvin

[lifepillar]

FYI, to avoid this issue my Solarized 8 fork does not define filetype-specific highlight groups by default. A setting is available if you still want to do so.

[ELLIOTTCABLE]

@lifepillar Why not include @auwsmit's solution as an option?

Speaking of: @auwsmit, may I include your answer in a related plugin? Care to release it under a license, so I can do so? (=

[auwsmit]

I don't mind at all, but that code almost entirely came from xolox's vim-colorscheme-switcher which is licensed under MIT, so I guess ask him, or include the MIT license with his copyright in your plugin to be safe. I'm not super familiar with license/copyright conventions.

[lifepillar]

@ELLIOTTCABLE I don't want to add bloat to the colorscheme. IMO, this issue can only be properly addressed in Vim core.