refactor(grid): change schar_T representation to be more compact

Previously, a screen cell would occupy 28+4=32 bytes per cell
as we always made space for up to MAX_MCO+1 codepoints in a cell.

As an example, even a pretty modest 50*80 screen would consume

50*80*2*32 = 256000, i e a quarter megabyte

With the factor of two due to the TUI side buffer, and even more when
using msg_grid and/or ext_multigrid.

This instead stores a 4-byte union of either:
- a valid UTF-8 sequence up to 4 bytes
- an escape char which is invalid UTF-8 (0xFF) plus a 24-bit index to a
  glyph cache

This avoids allocating space for huge composed glyphs _upfront_, while
still keeping rendering such glyphs reasonably fast (1 hash table lookup
+ one plain index lookup). If the same large glyphs are using repeatedly
on the screen, this is still a net reduction of memory/cache
consumption. The only case which really gets worse is if you blast
the screen full with crazy emojis and zalgo text and even this case
only leads to 4 extra bytes per char.

When only <= 4-byte glyphs are used, plus the 4-byte attribute code,
i e 8 bytes in total there is a factor of four reduction of memory use.
Memory which will be quite hot in cache as the screen buffer is scanned
over in win_line() buffer text drawing

A slight complication is that the representation depends on host byte
order. I've tested this manually by compling and running this
in qemu-s390x and it works fine. We might add a qemu based solution
to CI at some point.

1 files changed, 2 insertions, 14 deletions

diff --git a/src/nvim/mouse.c b/src/nvim/mouse.c
index b8c80cadf5..e35385dd43 100644
--- a/src/nvim/mouse.c
+++ b/src/nvim/mouse.c
@@ -1849,7 +1849,6 @@ static void mouse_check_grid(colnr_T *vcolp, int *flagsp)
   int click_grid = mouse_grid;
   int click_row = mouse_row;
   int click_col = mouse_col;
-  int mouse_char = ' ';
   int max_row = Rows;
   int max_col = Columns;
   bool multigrid = ui_has(kUIMultigrid);
@@ -1864,7 +1863,6 @@ static void mouse_check_grid(colnr_T *vcolp, int *flagsp)
   if (wp && mouse_row >= 0 && mouse_row < max_row
       && mouse_col >= 0 && mouse_col < max_col) {
     ScreenGrid *gp = multigrid ? &wp->w_grid_alloc : &default_grid;
-    int fdc = win_fdccol_count(wp);
     int use_row = multigrid && mouse_grid == 0 ? click_row : mouse_row;
     int use_col = multigrid && mouse_grid == 0 ? click_col : mouse_col;
 
@@ -1901,22 +1899,12 @@ static void mouse_check_grid(colnr_T *vcolp, int *flagsp)
         // concealed characters.
         *vcolp = col_from_screen;
       }
-
-      // Remember the character under the mouse, might be one of foldclose or
-      // foldopen fillchars in the fold column.
-      mouse_char = utf_ptr2char((char *)gp->chars[off]);
-    }
-
-    // Check for position outside of the fold column.
-    if (wp->w_p_rl ? click_col < wp->w_width_inner - fdc :
-        click_col >= fdc + (cmdwin_type == 0 ? 0 : 1)) {
-      mouse_char = ' ';
     }
   }
 
-  if (wp && mouse_char == wp->w_p_fcs_chars.foldclosed) {
+  if (col_from_screen == -2) {
     *flagsp |= MOUSE_FOLD_OPEN;
-  } else if (mouse_char != ' ') {
+  } else if (col_from_screen == -3) {
     *flagsp |= MOUSE_FOLD_CLOSE;
   }
 }