refactor(grid): change schar_T representation to be more compact

Previously, a screen cell would occupy 28+4=32 bytes per cell
as we always made space for up to MAX_MCO+1 codepoints in a cell.

As an example, even a pretty modest 50*80 screen would consume

50*80*2*32 = 256000, i e a quarter megabyte

With the factor of two due to the TUI side buffer, and even more when
using msg_grid and/or ext_multigrid.

This instead stores a 4-byte union of either:
- a valid UTF-8 sequence up to 4 bytes
- an escape char which is invalid UTF-8 (0xFF) plus a 24-bit index to a
  glyph cache

This avoids allocating space for huge composed glyphs _upfront_, while
still keeping rendering such glyphs reasonably fast (1 hash table lookup
+ one plain index lookup). If the same large glyphs are using repeatedly
on the screen, this is still a net reduction of memory/cache
consumption. The only case which really gets worse is if you blast
the screen full with crazy emojis and zalgo text and even this case
only leads to 4 extra bytes per char.

When only <= 4-byte glyphs are used, plus the 4-byte attribute code,
i e 8 bytes in total there is a factor of four reduction of memory use.
Memory which will be quite hot in cache as the screen buffer is scanned
over in win_line() buffer text drawing

A slight complication is that the representation depends on host byte
order. I've tested this manually by compling and running this
in qemu-s390x and it works fine. We might add a qemu based solution
to CI at some point.

1 files changed, 3 insertions, 2 deletions

diff --git a/src/nvim/msgpack_rpc/unpacker.c b/src/nvim/msgpack_rpc/unpacker.c
index c3b1022db2..37e32729cc 100644
--- a/src/nvim/msgpack_rpc/unpacker.c
+++ b/src/nvim/msgpack_rpc/unpacker.c
@@ -9,6 +9,7 @@
 #include "mpack/conv.h"
 #include "nvim/api/private/helpers.h"
 #include "nvim/ascii.h"
+#include "nvim/grid.h"
 #include "nvim/macros.h"
 #include "nvim/memory.h"
 #include "nvim/msgpack_rpc/channel_defs.h"
@@ -497,13 +498,13 @@ redo:
     if (g->icell == g->ncells - 1 && cellsize == 1 && cellbuf[0] == ' ' && repeat > 1) {
       g->clear_width = repeat;
     } else {
+      schar_T sc = schar_from_buf(cellbuf, cellsize);
       for (int r = 0; r < repeat; r++) {
         if (g->coloff >= (int)grid_line_buf_size) {
           p->state = -1;
           return false;
         }
-        memcpy(grid_line_buf_char[g->coloff], cellbuf, cellsize);
-        grid_line_buf_char[g->coloff][cellsize] = NUL;
+        grid_line_buf_char[g->coloff] = sc;
         grid_line_buf_attr[g->coloff++] = g->cur_attr;
       }
     }