The Terminal Programmer

Fixing Vim's Background Color Erase for 256-color tmux and GNU screen

Suraj N. Kurapati
(solution, vim, tmux, screen, color)

Problem

This is what you would see if you applied the Zenburn color scheme to Vim running inside tmux or GNU screen attached to the xterm-256color terminal, which provisions the TERM=xterm-256color environment mapping:

Flaky background in TERM=xterm-256color

Here, the terminal's background color is bleeding into Vim's, making Vim practically unusable for long editing sessions. But thankfully, this nuisance is easily deterred by running :set term=screen-256color in Vim or by restarting Vim under the TERM=screen-256color environment mapping:

Solid background in TERM=screen-256color

However, if you rely on Vim's ability to sense modifier keys—such as Shift, Control, and Alt—because you use them to define custom keyboard shortcuts like I do, then you would soon discover that Vim no longer recognizes them inside the screen-256color terminal.

Thus, you would be faced with a most unnerving dilemma:

  1. Do you forfeit your custom keyboard shortcuts to gain a proper background color by running Vim inside the screen-256color terminal?

  2. Do you suffer background color bleeding to retain your custom keyboard shortcuts by running Vim inside the xterm-256color terminal?

Alas, is it truly a binary choice? Must I really forgo my personal keyboard shortcuts in order to view my chosen Vim color scheme properly? No, I would not accept such a fate; there had to be a better way! >:O

Approach

I knew about Vim's ability to manipulate its host terminal through its t_Co setting, which lets you override how many colors Vim thinks its host terminal is capable of rendering. Thus, looking up :help t_Co in Vim revealed numerous terminal settings—which all happened to be listed in the terminal-options section of Vim's help documentation—that could be overridden, hopefully in the same way, to perhaps solve this problem.

However, there were too many settings in the documentation to practically investigate by hand, so I narrowed down the possibilities to the following eleven settings by searching for the words “color” and then “clear”:

OUTPUT CODES
  option  meaning
  t_AB    set background color (ANSI)
  t_AF    set foreground color (ANSI)
  t_cd    clear to end of screen
  t_ce    clear to end of line
  t_cl    clear screen
  t_Co    number of colors
  t_me    Normal mode (undoes t_mr, t_mb, t_md and color)
  t_op    reset to original color pair
  t_Sb    set background color
  t_Sf    set foreground color
  t_ut    clearing uses the current background color

I then observed the values of these settings in both environments so that I may (1) compare them to discover what makes Vim render its background color correctly in the screen-256color terminal and then (2) replicate those values to achieve the same effect in the xterm-256color terminal.

screen-256color observations:

t_cd=^[[J
t_ce=^[[K
t_cl=^[[H^[[J
t_AB=^[[%?%p1%{8}%<%t4%p1%d%e%p1%{16}%<%t10%p1%{8}%-%d%e48;5;%p1%d%;m
t_AF=^[[%?%p1%{8}%<%t3%p1%d%e%p1%{16}%<%t9%p1%{8}%-%d%e38;5;%p1%d%;m
t_Co=256
t_me=^[[0m
t_op=^[[39;49m
t_Sb=
t_Sf=
t_ut=

xterm-256color observations:

t_cd=^[[J
t_ce=^[[K
t_cl=^[[H^[[2J
t_AB=^[[%?%p1%{8}%<%t4%p1%d%e%p1%{16}%<%t10%p1%{8}%-%d%e48;5;%p1%d%;m
t_AF=^[[%?%p1%{8}%<%t3%p1%d%e%p1%{16}%<%t9%p1%{8}%-%d%e38;5;%p1%d%;m
t_Co=256
t_me=^[[m
t_op=^[[39;49m
t_Sb=
t_Sf=
t_ut=y

Comparing these observations, I found that xterm-256color differed from screen-256color by two insertions and one deletion. Moreover, these differences corresponded to three settings in particular: t_cl, t_me, and t_ut.

t_cd=^[[J
t_ce=^[[K
t_cl=^[[H^[[2J
t_AB=^[[%?%p1%{8}%<%t4%p1%d%e%p1%{16}%<%t10%p1%{8}%-%d%e48;5;%p1%d%;m
t_AF=^[[%?%p1%{8}%<%t3%p1%d%e%p1%{16}%<%t9%p1%{8}%-%d%e38;5;%p1%d%;m
t_Co=256
t_me=^[[0m
t_op=^[[39;49m
t_Sb=
t_Sf=
t_ut=y

Solution

One by one, I applied the screen-256color value of each differing setting to a Vim session that was running inside my desired xterm-256color terminal and then pressed Control-L to make Vim redraw itself. Changing t_cl and t_me had no effect but, thankfully, t_ut did the trick!

That makes sense because, per Vim documentation, not only does t_ut control whether Vim “uses the current background color” to clear the screen—also known as Background Color Erase, or BCE for short—but it also takes effect only when its value is a non-empty string.

In this case, Vim used BCE in xterm-256color because under that terminal, t_ut had value y: a non-empty string. Conversely, Vim did not use BCE in screen-256color because under that terminal, t_ut had no value.

Therefore, the solution is to simply clear Vim's t_ut value if Vim happens to be running inside a 256-color terminal. You can automate this by adding the following snippet to your Vim configuration file:

if &term =~ '256color'
  " Disable Background Color Erase (BCE) so that color schemes
  " work properly when Vim is used inside tmux and GNU screen.
  " See also http://snk.tuxfamily.org/log/vim-256color-bce.html
  set t_ut=
endif

Alternatives

Tom Ryder suggested that, rather than overriding the TERM environment variable, we should teach Vim how to recognize modifier keys produced by the xterm-256color terminal, as recommended in the tmux FAQ.

Following his reference to the “How do I make Ctrl-PgUp and Ctrl-PgDn work in vim?” section of the tmux FAQ and combining the instructions therein with this remedy (see step #4) yielded the following configuration snippets:

# ~/.tmux.conf
setw -g xterm-keys on
set -g default-terminal "screen-256color"

" ~/.vimrc
" Make Vim recognize xterm escape sequences for Page and Arrow
" keys combined with modifiers such as Shift, Control, and Alt.
" See http://www.reddit.com/r/vim/comments/1a29vk/_/c8tze8p
if &term =~ '^screen'
  " Page keys http://sourceforge.net/p/tmux/tmux-code/ci/master/tree/FAQ
  execute "set t_kP=\e[5;*~"
  execute "set t_kN=\e[6;*~"

  " Arrow keys http://unix.stackexchange.com/a/34723
  execute "set <xUp>=\e[1;*A"
  execute "set <xDown>=\e[1;*B"
  execute "set <xRight>=\e[1;*C"
  execute "set <xLeft>=\e[1;*D"
endif

These snippets teach Vim how to recognize modifier keys on the Page and Arrow keys (but not on the Function keys, see below) while retaining the TERM=screen-256color environment mapping so that Vim's background color is rendered properly, as the tmux FAQ recommends.

But what about the Function keys? Vim still cannot recognize them with modifier keys applied. To address this, Will Gray mentioned that he uses Matt Wozniski's terminalkeys.vim configuration, which is also available as a standalone Vim plugin on GitHub, to educate Vim accordingly.

As a result, Vim can now recognize all modifier keys producible by the xterm-256color terminal—even if Vim happens to be running under the TERM=screen-256color environment mapping provisioned by tmux or GNU screen.