sc-im background color issue

[poetaman]

Describe the bug

There is a wonderful ncurses based spreadsheet calculator program sc-im. It works well on my macOS terminal applications like iTerm2 and Apple's inbuilt Terminal.app, but not without issues on WezTerm.

1. Mouse's x-position seems to saturate beyond a certain width.
2. Upon scrolling beyond page boundary on the right, and scrolling back in, the application seems to lose its background color on the right edge.

None of these problems occur with iTerm2 or Terminal.app. I am using the latest build from yesterday night, and here's an mp4 video that shows the problem:

weztermncurses.mp4

[wez]

I can't get sc-im to work reasonably at all; the version in nix doesn't have mouse support and the one I tried to build locally just now thinks that the mac ncurses library is too old.

What I suspect is the root of the issue is that ncurses uses the old X11 mouse protocol which is limited in the number of columns that it can report, and is limited further because wezterm promises to only ever output utf-8 and that version of the mouse protocol can emit a byte sequence that is not valid utf-8.

I pushed an update to the wezterm.terminfo file that should instruct ncurses to use the newer SGR mouse reporting protocol that doesn't have those constraints.

This section of the docs has instructions on how to install and use wezterm's terminfo file:
https://wezfurlong.org/wezterm/faq.html?highlight=terminfo#how-do-i-enable-undercurl-curly-underlines

[poetaman]

@wez Awesome, this fixes the mouse issue, but the other issue (background color being compromised) remains.

Do you use homebrew? If yes, then these steps will help (for basic setup, it doesn't link it to liblua, but thats not needed for this testing):

>>export PKG_CONFIG_PATH="<path_to_homebrew_installdir>/lib:<path_to_homebrew_installdir>/lib/pkgconfig:/usr/local/lib:/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH"
>> brew install ncurses
>> brew install sc-im

Now check if it has linked to the latest "ncursesw" library using
>> otool -L <full path to homebrew's sc-im>

On my system (macOS ARM64/Apple M1) it gives:
>>otool -L /opt/homebrew/Cellar/sc-im/0.8.1/bin/sc-im
/opt/homebrew/Cellar/sc-im/0.8.1/bin/sc-im:
        /opt/homebrew/opt/ncurses/lib/libncursesw.6.dylib (compatibility version 6.0.0, current version 6.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)

Before testing sc-im for the background issue I show in video, set its config file (~/.config/sc-im/scimrc) to:

set autocalc
set numeric
set numeric_zero
set numeric_decimal
set newline_action=j
set truncate
set xlsx_readformulas

# Papercolor dark theme
REDEFINE_COLOR "WHITE" 208 208 208
REDEFINE_COLOR "BLACK" 28 28 28
REDEFINE_COLOR "RED" 255 149 128
REDEFINE_COLOR "GREEN" 175 215 0
REDEFINE_COLOR "YELLOW" 255 175 95
REDEFINE_COLOR "BLUE" 95 175 215
REDEFINE_COLOR "CYAN" 0 175 175
REDEFINE_COLOR "MAGENTA" 255 95 175
DEFINE_COLOR "comment" 0 135 135
DEFINE_COLOR "altbackground" 88 88 88
DEFINE_COLOR "infofg" 215 175 95
DEFINE_COLOR "errorbg" 175 0 0

color "type=DEFAULT fg=WHITE bg=BLACK bold=0 italic=0"
color "type=NORMAL fg=WHITE bg=BLACK bold=0 italic=0"
color "type=WELCOME fg=CYAN bg=BLACK bold=0"
color "type=MODE fg=infofg bg=BLACK bold=0"
color "type=GRID_EVEN fg=WHITE bg=BLACK"
color "type=GRID_ODD fg=WHITE bg=BLACK"
color "type=CELL_ID fg=WHITE bg=BLACK bold=0"
color "type=CELL_FORMAT fg=comment bg=BLACK bold=0"
color "type=CELL_CONTENT fg=WHITE bg=BLACK bold=0"
color "type=INFO_MSG fg=infofg bg=BLACK bold=0"
color "type=ERROR_MSG fg=WHITE bg=errorbg bold=0"
color "type=INPUT fg=BLUE bg=BLACK"
color "type=CELL_ERROR fg=WHITE bg=errorbg bold=0"
color "type=HEADINGS fg=comment bg=BLACK bold=0"
color "type=HEADINGS_ODD fg=comment bg=BLACK bold=0"
color "type=CELL_SELECTION fg=WHITE bg=altbackground bold=1"
color "type=CELL_SELECTION_SC fg=WHITE bg=altbackground bold=1"
color "type=NUMB fg=YELLOW bg=BLACK bold=0"
color "type=CELL_NEGATIVE fg=MAGENTA bg=BLACK bold=0"
color "type=EXPRESSION fg=GREEN bg=BLACK bold=0"
# To not italicize strings, replace italic=1 with italic=0 on following line
color "type=STRG fg=BLUE bg=BLACK bold=0 italic=1"
color "type=DATEF fg=YELLOW bg=BLACK"
color "type=HELP_HIGHLIGHT fg=BLACK bg=WHITE"

I know it could be a lot to ask to do, though I think sc-im is worth the investment. It is the program that has made me like the idea of spreadsheets, and I am volunteering with testing & themes to its repository.

[wez]

Could you see if this is still an issue with the current nightly?

[poetaman]

@wez Just checked with nightly of WezTerm & latest dev branch build of sc-im. Yep it still happens. In my observation, the color error happens in the region of the height of the spreadsheet table. It does not happen in iTerm2 or Alacritty... It also happens inside tmux inside WezTerm, and does not happen inside tmux inside iTerm2 or Alacritty.

[wez]

Please capture a screen recording that focuses on the event where the background color gets broken; that will help me understand what is being sent to the terminal. If you can set the size to 80x24 that will help me replay the recording on other terminals without having to mess around trying to get the same dimensions :)

[poetaman]

Here's a video comparing the typescript I sent you on discord. To the left is WezTerm, and to the right is iTerm2:

Screen.Recording.2021-09-03.at.2.43.43.PM.mov

[wez]

I think this is fixed now in main!

[poetaman]

@wez Yes sc-im background issue is fixed. Btw there is another orthogonal issue with tmux background color which existed before this fix and which still persists to exist. I thought it could be same issue so had not filed it. Here's the video:

If you would like I can file it in a separate issue...

tmux_wezterm.mp4

Steps: 1) edit your tmux conf file to set some different background color for tmux active window, and window. 2) Then open tmux by running >>tmux (it will create new session), 3) detach the client, repeat steps 2-3 couple of times till you see the issue in above video.

tmux set -g window-style "bg=<some_hex_value>, fg=<some_hex_value>"
tmux set -g window-active-style "bg=<some_hex_value>, fg=<some_hex_value>"

[wez]

Please file a separate issue for that!
Let's close this one out :)

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.