fix(tui): add fixups for hterm family #19078

Problem
-------

In #19040, I reported two things that started happening somewhen in the
last three months when using neovim in hterm (the Chrome Secure Shell
terminal):

 1. Under certain circumstances, the window title (set by nvim
    [i0]) would appear over the line I was typing, corrupting the screen.
 2. If I changed my $TERM from xterm-256color to the new hterm-256color
    (available since ncurses >=20210320), the window title corruption
    was gone, but pane scrolling was broken.

Both problems are due to changes in the termcap files, their source of
truth being the ncurses project. See "Timeline of ncurses changes" below
for details.

Cause: title corruption
-----------------------

The title corruption when using hterm + TERM=xterm-256color can be
explained by event #4 (ncurses 2022-03-12) in the ncurses timeline:

The xterm-256color termcap file gained status line termcodes in ncurses
2022-03-12. These termcodes are used by Neovim to set the title when.
hterm does not have a status line. Due to ncurses versions earlier than
2022-03-12 missing the xterm status line capability, Neovim manually
fixed up [t0] the terminfo file if $TERM was xterm-256color. So if
before Neovim manually added fsl/tsl capabilties, and after they were
in the termcap file, why did hterm suddenly start getting corruption?
The answer is that the termcodes for these capabilties are different
when Neovim fixes them up, versus the one in the new termcap database:

   fsl=\E[0$}                  // from xterm-256color
   tsl=\E[2$~\E[1$}\E[%i%p1%d` // from xterm-256color

   fsl=\x07                    // patched by Neovim
   tsl=\x1b]0;                 // patched by Neovim

hterm ignores the latter, but corrupts the screen with the former.

Solution: Make hterm users set hterm-256color, which lacks the new
fsl/tsl codes. Also, to reduce superfluous work, stop patching in this
capability when hterm is detected (even if hterm would ignore the
patched version).

Cause: pane corruption
----------------------

The pane corruption when using hterm + TERM=hterm-256color, but NOT when
using hterm + TERM=xterm-256color can be explained by:

 - Neovim uses DECSLRM when available [p1] for performant scrolling.
 - Both the hterm-256color and xterm-256color termcap databases
   advertise support for DECSLRM (ncurses timeline #1, #2 and #3).
 - hterm does not support DESCLRM [p2] (note: it does support DESCTBM for
   top/bottom scrolling, but it's broken [p3] and not used by Neovim)
 - xterm-alikes that are not real xterm generally don't support DECSLRM
   either, so Neovim patches it out [p4].

When using hterm-256color, hterm is no longer considered an xterm-alike
by Neovim. As a result, DECSLRM is not cleared. hterm does not support
it, so corruption ensues.

This is a problem with the hterm-256color termcap file, but we're stuck
with it so the best we can do is patch over it.

Timeline of ncurses changes
---------------------------

 1. 2019-05-19: Part of the DECSLRM capability (smglr AKA set_lr_margin)
    added to vt420+lrmm, which xterm-256color inherits [n1]
 2. 2021-03-20: hterm-256color added, inheriting xterm-256colors. [n2]
 3. 2021-09-25: The *parm versions of smglr (AKA set_lr_margin) were
    added to vt420+lrmm [n3]. Namely:
     1. smglp AKA set_left_margin_parm, and
     2. smgrp AKA set_right_margin_parm
 4. 2022-03-12: (new) codes for fsl, bsl and tsl added to xterm (add
    dec+sl to xterm-new, per patch #371 -TD) [n4]

Fixes #19040.

[i0]: https://github.com/neovim/neovim/blob/3a4fa22badc5595afc0a994ead965ff32ccf6c76/src/nvim/tui/tui.c#L1377
[t0]: https://github.com/neovim/neovim/blob/3a4fa22badc5595afc0a994ead965ff32ccf6c76/src/nvim/tui/tui.c#L1728,L1729
[p1]: https://github.com/neovim/neovim/blob/3a4fa22badc5595afc0a994ead965ff32ccf6c76/src/nvim/tui/tui.c#L1196
[p2]: https://bugs.chromium.org/p/chromium/issues/detail?id=1175065&q=component%3APlatform%3EApps%3EDefault%3EHterm
[p3]: https://bugs.chromium.org/p/chromium/issues/detail?id=1298796&q=component%3APlatform%3EApps%3EDefault%3EHterm
[p4]: https://github.com/neovim/neovim/blob/3a4fa22badc5595afc0a994ead965ff32ccf6c76/src/nvim/tui/tui.c#L1740-L1752
[n1]: https://github.com/mirror/ncurses/commit/8f6d94b8d6211a2323caef53fa4c96c475ec9a6#diff-01544c577762d3308a1d232aa7afc79acf64b9a5057f88a004df82fda89549b7R2742
[n2]: https://github.com/mirror/ncurses/commit/c2650100f80134924eda8f22cd7cc1d1d919ee3d#diff-01544c577762d3308a1d232aa7afc79acf64b9a5057f88a004df82fda89549b7R5907
[n3]: https://github.com/mirror/ncurses/commit/f6b436c4fb50275df43ea10ba9c744fe195a327d#diff-01544c577762d3308a1d232aa7afc79acf64b9a5057f88a004df82fda89549b7R2842
[n4]: https://github.com/mirror/ncurses/commit/8bf8c836fe2c28238adbd62e71f03272efe155ad#diff-01544c577762d3308a1d232aa7afc79acf64b9a5057f88a004df82fda89549b7R4828

Signed-off-by: Nicolas Hillegeer <nicolas@hillegeer.com>

1 files changed, 17 insertions, 6 deletions

diff --git a/src/nvim/tui/tui.c b/src/nvim/tui/tui.c
index 20cf1ed586..e2289eb9ce 100644
--- a/src/nvim/tui/tui.c
+++ b/src/nvim/tui/tui.c
@@ -1642,6 +1642,7 @@ static void patch_terminfo_bugs(TUIData *data, const char *term, const char *col
   bool xterm = terminfo_is_term_family(term, "xterm")
                // Treat Terminal.app as generic xterm-like, for now.
                || nsterm;
+  bool hterm = terminfo_is_term_family(term, "hterm");
   bool kitty = terminfo_is_term_family(term, "xterm-kitty");
   bool linuxvt = terminfo_is_term_family(term, "linux");
   bool bsdvt = terminfo_is_bsd_console(term);
@@ -1705,7 +1706,7 @@ static void patch_terminfo_bugs(TUIData *data, const char *term, const char *col
     unibi_set_bool(ut, unibi_back_color_erase, false);
   }
 
-  if (xterm) {
+  if (xterm || hterm) {
     // Termit, LXTerminal, GTKTerm2, GNOME Terminal, MATE Terminal, roxterm,
     // and EvilVTE falsely claim to be xterm and do not support important xterm
     // control sequences that we use.  In an ideal world, these would have
@@ -1714,9 +1715,13 @@ static void patch_terminfo_bugs(TUIData *data, const char *term, const char *col
     // treatable as xterm.
 
     // 2017-04 terminfo.src lacks these.  Xterm-likes have them.
-    unibi_set_if_empty(ut, unibi_to_status_line, "\x1b]0;");
-    unibi_set_if_empty(ut, unibi_from_status_line, "\x07");
-    unibi_set_if_empty(ut, unibi_set_tb_margin, "\x1b[%i%p1%d;%p2%dr");
+    if (!hterm) {
+      // hterm doesn't have a status line.
+      unibi_set_if_empty(ut, unibi_to_status_line, "\x1b]0;");
+      unibi_set_if_empty(ut, unibi_from_status_line, "\x07");
+      // TODO(aktau): patch this in when DECSTBM is fixed (https://crbug.com/1298796)
+      unibi_set_if_empty(ut, unibi_set_tb_margin, "\x1b[%i%p1%d;%p2%dr");
+    }
     unibi_set_if_empty(ut, unibi_enter_italics_mode, "\x1b[3m");
     unibi_set_if_empty(ut, unibi_exit_italics_mode, "\x1b[23m");
 
@@ -1727,6 +1732,9 @@ static void patch_terminfo_bugs(TUIData *data, const char *term, const char *col
       unibi_set_if_empty(ut, unibi_set_right_margin_parm, "\x1b[%i;%p2%ds");
     } else {
       // Fix things advertised via TERM=xterm, for non-xterm.
+      //
+      // TODO(aktau): stop patching this out for hterm when it gains support
+      // (https://crbug.com/1175065).
       if (unibi_get_str(ut, unibi_set_lr_margin)) {
         ILOG("Disabling smglr with TERM=xterm for non-xterm.");
         unibi_set_str(ut, unibi_set_lr_margin, NULL);
@@ -1875,6 +1883,8 @@ static void patch_terminfo_bugs(TUIData *data, const char *term, const char *col
         && ((xterm && !vte_version)  // anything claiming xterm compat
             // per MinTTY 0.4.3-1 release notes from 2009
             || putty
+            // per https://chromium.googlesource.com/apps/libapps/+/a5fb83c190aa9d74f4a9bca233dac6be2664e9e9/hterm/doc/ControlSequences.md
+            || hterm
             // per https://bugzilla.gnome.org/show_bug.cgi?id=720821
             || (vte_version >= 3900)
             || (konsolev >= 180770)  // #9364
@@ -1959,6 +1969,7 @@ static void augment_terminfo(TUIData *data, const char *term, long vte_version,
   bool xterm = terminfo_is_term_family(term, "xterm")
                // Treat Terminal.app as generic xterm-like, for now.
                || nsterm;
+  bool hterm = terminfo_is_term_family(term, "hterm");
   bool bsdvt = terminfo_is_bsd_console(term);
   bool dtterm = terminfo_is_term_family(term, "dtterm");
   bool rxvt = terminfo_is_term_family(term, "rxvt");
@@ -1988,7 +1999,7 @@ static void augment_terminfo(TUIData *data, const char *term, long vte_version,
                                                            "ext.resize_screen",
                                                            "\x1b[8;%p1%d;%p2%dt");
   }
-  if (putty || xterm || rxvt) {
+  if (putty || xterm || hterm || rxvt) {
     data->unibi_ext.reset_scroll_region = (int)unibi_add_ext_str(ut,
                                                                  "ext.reset_scroll_region",
                                                                  "\x1b[r");
@@ -2045,7 +2056,7 @@ static void augment_terminfo(TUIData *data, const char *term, long vte_version,
       // would use a tmux control sequence and an extra if(screen) test.
       data->unibi_ext.set_cursor_color =
         (int)unibi_add_ext_str(ut, NULL, TMUX_WRAP(tmux, "\033]Pl%p1%06x\033\\"));
-    } else if ((xterm || rxvt || tmux || alacritty)
+    } else if ((xterm || hterm || rxvt || tmux || alacritty)
                && (vte_version == 0 || vte_version >= 3900)) {
       // Supported in urxvt, newer VTE.
       data->unibi_ext.set_cursor_color = (int)unibi_add_ext_str(ut, "ext.set_cursor_color",