Styled underlines

[eatgrass]

add underline styles

<ESC>[4:0m  # no underline
<ESC>[4:1m  # straight underline
<ESC>[4:2m  # double underline
<ESC>[4:3m  # curly underline
<ESC>[4:4m  # dotted underline
<ESC>[4:5m  # dashed underline
<ESC>[4m    # straight underline (for backwards compat)
<ESC>[24m   # no underline (for backwards compat)
<ESC>[58...m # underline color
<ESC>[59m // reset underline color

Term kitty

Term Alacritty

iTerm2

printf '\e[4:1mHello, underline!\n\e[m\e[4:2mHello, double-underline!\n\e[m\e[4:3mHello, under-curl!\n\e[m\e[4:4mHello, under-dotted!\n\e[m\e[4:5mHello, under-dashed!\n\e[m\e[58;2;255;0;0m\e[4:1mHello, underline!\n\e[m\e[58;2;255;0;0m\e[4:2mHello, double-underline!\n\e[m\e[58;2;255;0;0m\e[4:3mHello, under-curl!\n\e[m\e[58;2;255;0;0m\e[4:4mHello, under-dotted!\n\e[m\e[58;2;255;0;0m\e[4:5mHello, under-dashed!\n\e[m
'

[thomasaarholt]

This is great! Came here after thinking there was something wrong with my setup, since I've been using helix (a vim like editor), and suddenly started losing red-underscore-errors in my python environment. I'd simultaneously started using zellij, which was the reason.

I checked out this branch and built zellij, and this works nicely!

Here are some screenshots, using the wezterm terminal emulator:
Without this PR:

With this PR:

[Whax]

Any chance to see this PR merged ? :)

[mike-lloyd03]

Is there anything the community could do to help get this merged? Perhaps some testing on the PR branch?

[imsnif]

Hey, thanks for this.

Unfortunately, this protocol repurposes some old CSI codes that do different things in unsupported terminals. For example, this will cause xterm to show a blue background: echo -e "\033[58;2;224;175;104mI should not be blue"

I'd be open to merging this if it's only rendered for supported terminals (I think this can be detected through TERMINFO).

[imsnif]

Actually - to be fair to the protocol, I think this happens because xterm eagerly parses the SGR parameters, discarding ones it does not know and interpreting the rest. In any case though, we can't broadcast this to unsupported terminals because I guess this eager interpretation is not "wrong" per-se. So, gotta identify the attribute (through zellij-client, because this is the user's terminal) from terminfo.

[mike-lloyd03]

Could it be set as a configuration option, default off? So users can opt in if they want it and leave it off if they don't.

[imsnif]

I get the value this would have in simplifying things, but I'd rather not involve users in this decision.

[eatgrass]

@imsnif Thanks for your patient explanation.

I'd be open to merging this if it's only rendered for supported terminals (I think this can be detected through TERMINFO).

It seems the extended capabilities didn't go into the term database, for example, I noticed the full capability list of kitty doesn't contain those underline styles, and they are also not present in the terminfo crate. maybe we should find another way to do this, let me know what you think

#	Reconstructed via infocmp from file: /Applications/kitty.app/Contents/Resources/kitty/terminfo/78/xterm-kitty
xterm-kitty|KovIdTTY,
	auto_right_margin, can_change, eat_newline_glitch,
	has_meta_key, has_status_line, move_insert_mode,
	move_standout_mode, no_pad_char, prtr_silent,
	columns#80, init_tabs#8, lines#24, max_colors#256,
	max_pairs#32767,
	acs_chars=++\054\054--..00``aaffgghhiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
	back_tab=\E[Z, bell=^G, carriage_return=^M,
	change_scroll_region=\E[%i%p1%d;%p2%dr,
	clear_all_tabs=\E[3g, clear_screen=\E[H\E[2J,
	clr_bol=\E[1K, clr_eol=\E[K, clr_eos=\E[J,
	column_address=\E[%i%p1%dG,
	cursor_address=\E[%i%p1%d;%p2%dH, cursor_down=^J,
	cursor_home=\E[H, cursor_invisible=\E[?25l,
	cursor_left=^H, cursor_normal=\E[?12h\E[?25h,
	cursor_right=\E[C, cursor_up=\E[A,
	cursor_visible=\E[?12;25h, delete_character=\E[P,
	delete_line=\E[M, dis_status_line=\E]2;\E\,
	enter_alt_charset_mode=\E(0, enter_am_mode=\E[?7h,
	enter_bold_mode=\E[1m, enter_ca_mode=\E[?1049h,
	enter_dim_mode=\E[2m, enter_insert_mode=\E[4h,
	enter_italics_mode=\E[3m, enter_reverse_mode=\E[7m,
	enter_standout_mode=\E[7m,
	enter_underline_mode=\E[4m, erase_chars=\E[%p1%dX,
	exit_alt_charset_mode=\E(B, exit_am_mode=\E[?7l,
	exit_attribute_mode=\E(B\E[m, exit_ca_mode=\E[?1049l,
	exit_insert_mode=\E[4l, exit_italics_mode=\E[23m,
	exit_standout_mode=\E[27m,
	exit_underline_mode=\E[24m,
	flash_screen=\E[?5h$<100/>\E[?5l,
	from_status_line=^G,
	initialize_color=\E]4;%p1%d;rgb\072%p2%{255}%*%{1000}%/%2.2X/%p3%{255}%*%{1000}%/%2.2X/%p4%{255}%*%{1000}%/%2.2X\E\,
	insert_line=\E[L, key_a1=, key_a3=, key_backspace=\177,
	key_beg=\EOE, key_btab=\E[Z, key_c1=, key_c3=,
	key_dc=\E[3~, key_down=\EOB, key_end=\EOF, key_f1=\EOP,
	key_f10=\E[21~, key_f11=\E[23~, key_f12=\E[24~,
	key_f13=\E[1;2P, key_f14=\E[1;2Q, key_f15=\E[13;2~,
	key_f16=\E[1;2S, key_f17=\E[15;2~, key_f18=\E[17;2~,
	key_f19=\E[18;2~, key_f2=\EOQ, key_f20=\E[19;2~,
	key_f21=\E[20;2~, key_f22=\E[21;2~, key_f23=\E[23;2~,
	key_f24=\E[24;2~, key_f25=\E[1;5P, key_f26=\E[1;5Q,
	key_f27=\E[13;5~, key_f28=\E[1;5S, key_f29=\E[15;5~,
	key_f3=\EOR, key_f30=\E[17;5~, key_f31=\E[18;5~,
	key_f32=\E[19;5~, key_f33=\E[20;5~, key_f34=\E[21;5~,
	key_f35=\E[23;5~, key_f36=\E[24;5~, key_f37=\E[1;6P,
	key_f38=\E[1;6Q, key_f39=\E[13;6~, key_f4=\EOS,
	key_f40=\E[1;6S, key_f41=\E[15;6~, key_f42=\E[17;6~,
	key_f43=\E[18;6~, key_f44=\E[19;6~, key_f45=\E[20;6~,
	key_f46=\E[21;6~, key_f47=\E[23;6~, key_f48=\E[24;6~,
	key_f49=\E[1;3P, key_f5=\E[15~, key_f50=\E[1;3Q,
	key_f51=\E[13;3~, key_f52=\E[1;3S, key_f53=\E[15;3~,
	key_f54=\E[17;3~, key_f55=\E[18;3~, key_f56=\E[19;3~,
	key_f57=\E[20;3~, key_f58=\E[21;3~, key_f59=\E[23;3~,
	key_f6=\E[17~, key_f60=\E[24;3~, key_f61=\E[1;4P,
	key_f62=\E[1;4Q, key_f63=\E[13;4~, key_f7=\E[18~,
	key_f8=\E[19~, key_f9=\E[20~, key_help=, key_home=\EOH,
	key_ic=\E[2~, key_left=\EOD, key_mouse=\E[M,
	key_npage=\E[6~, key_ppage=\E[5~, key_right=\EOC,
	key_sbeg=\E[1;2E, key_sdc=\E[3;2~, key_send=\E[1;2F,
	key_sf=\E[1;2B, key_shome=\E[1;2H, key_sic=\E[2;2~,
	key_sleft=\E[1;2D, key_snext=\E[6;2~,
	key_sprevious=\E[5;2~, key_sr=\E[1;2A,
	key_sright=\E[1;2C, key_undo=, key_up=\EOA,
	keypad_local=\E[?1l, keypad_xmit=\E[?1h,
	orig_colors=\E]104\007, orig_pair=\E[39;49m,
	parm_dch=\E[%p1%dP, parm_delete_line=\E[%p1%dM,
	parm_down_cursor=\E[%p1%dB, parm_ich=\E[%p1%d@,
	parm_index=\E[%p1%dS, parm_insert_line=\E[%p1%dL,
	parm_left_cursor=\E[%p1%dD,
	parm_right_cursor=\E[%p1%dC, parm_rindex=\E[%p1%dT,
	parm_up_cursor=\E[%p1%dA,
	repeat_char=%p1%c\E[%p2%{1}%-%db,
	reset_1string=\E]\E\\Ec, restore_cursor=\E8,
	row_address=\E[%i%p1%dd, save_cursor=\E7,
	scroll_forward=^J, scroll_reverse=\EM,
	set_a_background=\E[%?%p1%{8}%<%t4%p1%d%e%p1%{16}%<%t10%p1%{8}%-%d%e48;5;%p1%d%;m,
	set_a_foreground=\E[%?%p1%{8}%<%t3%p1%d%e%p1%{16}%<%t9%p1%{8}%-%d%e38;5;%p1%d%;m,
	set_attributes=%?%p9%t\E(0%e\E(B%;\E[0%?%p6%t;1%;%?%p2%t;4%;%?%p1%p3%|%t;7%;%?%p4%t;5%;%?%p7%t;8%;m,
	set_tab=\EH, tab=^I, to_status_line=\E]2;,
	user6=\E[%i%d;%dR, user7=\E[6n,
	user8=\E[?%[;0123456789]c, user9=\E[c,

[imsnif]

They do appear in the extended terminfo user-defined capabilities of Kitty (for me at least) with infocmp -x:

$ infocmp -x
#       Reconstructed via infocmp from file: /usr/lib/kitty/terminfo/x/xterm-kitty
xterm-kitty|KovIdTTY,
        am, ccc, hs, km, mc5i, mir, msgr, npc, xenl, Su, Tc, fullkbd, # The "Su" here
        colors#0x100, cols#80, it#8, lines#24, pairs#0x7fff,
...

However, they do not appear in a version of Alacritty that supports this for me and I guess this might happen with other terminals as well. It's unfortunate that this protocol was designed in a way that encourages non-backwards compatibility (eg. I don't understand why a private escape sequence was not used instead), but I guess that's life.

Considering this, I'd be willing to merge this along with a disable_styled_underlines config variable as suggested above (making this opt-out rather than opt-in because I think the case of this causing trouble for modern terminal users is by far the less common one).

[imsnif]

Hey @eatgrass - any updates? All good if you don't want to work on this further, but in that case please let us know so that someone else can pick this up if they want.

[eatgrass]

I'm sorry for not getting back to you sooner. I've been overwhelmed with my work recently, so I would appreciate it if anyone could help with this feature.

[imsnif]

Thanks for letting us know @eatgrass - this is now up for grabs if anyone wants to see this through. To recap: what we need is a zellij configuration to disable these, something like ansi_underlines false in the Zellij configuration. You can take a look at how auto_layout works and follow it until Screen, and then pass it down to TerminalPane and PluginPane when they are constructed.

[mike-lloyd03]

I'm working on getting the config option ansi_underlines plugged in. I'd like to finish out this PR but I'm not sure how to push commits to it. Should I just fork and make a new PR?

[mike-lloyd03]

@eatgrass I think you'll have to grant me permissions to push commits to your fork branch.

[eatgrass]

@eatgrass I think you'll have to grant me permissions to push commits to your fork branch.

sure, thank you in advance

[mike-lloyd03]

@eatgrass Thanks!

@imsnif I pushed up my changes which create the new config variable (ansi_underlines) and pass it from Screen to Tab to TerminalPane/PluginPane and finally to Grid. I also implemented the logic on CharacterStyles to handle the ansi_underlines field but I'm not sure how to connect this to the actual config variable.

Also, it looks like I may need to rebase this with main. I'll get that done tonight as well.

[imsnif]

Great to hear @mike-lloyd03 ! To connect this to the actual config, take a look at https://github.com/zellij-org/zellij/blob/main/zellij-utils/src/kdl/mod.rs (you can follow the example of auto_layout - I think it's mostly just copy/pasting and changing the name).

Let me know when this is done and I'll give it a whirl.

[mike-lloyd03]

@imsnif I think I got it. But this really blew up. I had to add ansi_underlines: bool to the foreground_color and background_color in zellij-server/src/ui/pane_boundaries_frame.rs which required a very large amount of changes.

But at the end of the day, it works (on my machine). Let me know if you need any changes.

Would it be possible to refactor the config implementation to read the config file on initialization and set it as a global static using once_cell::sync::Lazy? Then we wouldn't have to pass config variables down through the stack. I'm sure that much easier said than done, but I'd like to hear your thoughts on that approach.

[mike-lloyd03]

Okay I think we're good to go here. And thanks for catching my glaring error on the single underline.

Everything else has been addressed.

[imsnif]

Perfect. Looks great. I also did some more manual tests for this and everything seems to be in order.

Thanks to everyone involved for the work on this!

[danillos]

I'm having this issue:

Using kitty + Zellij + Helix, the curl doesn't show for me inside of the editor
Using kitty + Helix, the curl shows for me inside of the editor

Where is this issue, Zellij or Helix?

[sereinity]

Also no curls on alacritty + zellij + nvim (and with Helix)

[jscarrott]

Also doesn't work for me in nvim or helix. I did debug helix though and can see the ansi codes being written so not sure what is going on. I tried to debug zellij to see what it was seeing but couldn't work out how to debug it as none of the breakpoints were getting hit, presumable it sping up a server instance that isn't being debugged.

[imsnif]

ping @eatgrass @mike-lloyd03 - any hints?

[danillos]

One of the Helix maintainers said it some time ago, helix-editor/helix#6510 (comment)

So it may not be related on the render, but to identify if can render or not it.

But his suggestion there didn't work for me. so I don't know if this information is helpful or not.

[jscarrott]

One of the Helix maintainers said it some time ago, helix-editor/helix#6510 (comment)

So it may not be related on the render, but to identify if can render or not it.

But his suggestion there didn't work for me. so I don't know if this information is helpful or not.

Undercurls were forced on for me, and I debugged to make sure that I could see the ansi code being written. The helix dev's comment was in the context of zellij not supporting underlines which was true until this PR.

[mike-lloyd03]

I'm having the same problem. Undercurls work in Zellij outside nvim and helix but not inside. Unfortunately, in my testing, I was testing at the terminal prompt and never tested in an editor. My apologies.

The problem is with Zellij not nvim or helix. Specifically with how we're handling the configuration variable. Styled underlines worked correctly with @eatgrass 's original changes and broke with my additions. I'll dig into this today and hopefully get a new PR going to fix it.

[imsnif]

Thanks @mike-lloyd03 ! I'm a little puzzled by this myself. I guess there's somewhere we missed passing the config to CharacterStyles?

[mike-lloyd03]

I'm not sure, the problem appears in impl Display for CharacterStyles.

For some reason, self.styled_underlines_enabled is evaluated to true when we're at the command prompt but false when we're inside an editor. If I comment out the two places where we have if self.styled_underlines_enabled { ... in this impl, it works as expected.

Also weird, if I override Default for CharacterStyles so that styled_underlines_enabled is true, it still doesn't work. Like something is explicitly setting it to false.

[imsnif]

Alright, looks like you're pin-pointing it. I'm dealing with some other fires rn - can I trust you to figure it out?

[mike-lloyd03]

Yep

[mike-lloyd03]

The problem is with RESET_STYLES setting styled_underlines_enabled to false. If I change it to true, the problem is resolved but this would probably create issues on unsupported terminals. I'm investigating a way to retain the styled_underlines_enabled value when RESET_STYLES is used.

[imsnif]

Ohhh... nice catch. But also - ouch, this is a harsh one. iirc we use it profusely in lots of places. My instinct tells me the only way to solve this would be to turn it into a function of some sort and pass the config to it, but maybe you have a different idea...

[mike-lloyd03]

Yeah I'm leaning in that direction as well. I'm trying to pinpoint the place where it's critical and I'll decide from there.

[mike-lloyd03]

Got it working. #2918

[mmgeorge]

EDIT: Sorry, really should have just put this in an issue: #3262

---

Thanks for the work on this! Undercurls are working great, but I seem to be hitting some problems with underline colors in WSL + Window Terminal 1.19/1.20:

echo -e "\x1b[58:5:32m\x1b[4mtest\x1b[59m\x1b[24m"
echo -e "\x1b[4:3mTest\x1b[24m "