refactor(grid): make screen rendering more multibyte than ever before

Problem: buffer text with composing chars are converted from UTF-8
to an array of up to seven UTF-32 values and then converted back
to UTF-8 strings.

Solution: Convert buffer text directly to UTF-8 based schar_T values.

The limit of the text size is now in schar_T bytes, which is currently
31+1 but easily could be raised as it no longer multiplies the size
of the entire screen grid when not used, the full size is only required
for temporary scratch buffers.

Also does some general cleanup to win_line text handling, which was
unnecessarily complicated due to multibyte rendering being an "opt-in"
feature long ago. Nowadays, a char is just a char, regardless if it consists
of one ASCII byte or multiple bytes.

4 files changed, 20 insertions, 5 deletions

diff --git a/runtime/doc/mbyte.txt b/runtime/doc/mbyte.txt
index aedef87a09..0a7e0baad3 100644
--- a/runtime/doc/mbyte.txt
+++ b/runtime/doc/mbyte.txt
@@ -646,7 +646,8 @@ widespread as file format.
 A composing or combining character is used to change the meaning of the
 character before it.  The combining characters are drawn on top of the
 preceding character.
-Up to six combining characters can be displayed.
+Too big combined characters cannot be displayed, but they can still be
+inspected using the |g8| and |ga| commands described below.
 When editing text a composing character is mostly considered part of the
 preceding character.  For example "x" will delete a character and its
 following composing characters by default.
diff --git a/runtime/doc/news.txt b/runtime/doc/news.txt
index 2f48ebfeff..cb3220a630 100644
--- a/runtime/doc/news.txt
+++ b/runtime/doc/news.txt
@@ -294,6 +294,13 @@ The following changes to existing APIs or features add new behavior.
   Note that syntax highlighting of code examples requires a matching parser
   and may be affected by custom queries.
 
+• Support for rendering multibyte characters using composing characters has been
+  enhanced. The maximum limit have been increased from 1+6 codepoints to
+  31 bytes, which is guaranteed to fit all chars from before but often more.
+
+  NOTE: the regexp engine still has a hard-coded limit of considering
+  6 composing chars only.
+
 ==============================================================================
 REMOVED FEATURES                                                 *news-removed*
 
diff --git a/runtime/doc/vim_diff.txt b/runtime/doc/vim_diff.txt
index 05d7e5feb9..5e09cc2481 100644
--- a/runtime/doc/vim_diff.txt
+++ b/runtime/doc/vim_diff.txt
@@ -722,9 +722,16 @@ Options:
 <
   *'macatsui'*
   *'maxcombine'* *'mco'*
-    Nvim always displays up to 6 combining characters.  You can still edit
-    text with more than 6 combining characters, you just can't see them.
-    Use |g8| or |ga|.  See |mbyte-combining|.
+    Nvim counts maximum character sizes in bytes, not codepoints. This is
+    guaranteed to be big enough to always fit all chars properly displayed
+    in vim with 'maxcombine' set to 6.
+
+    You can still edit text with larger characters than fits in the screen buffer,
+    you just can't see them. Use |g8| or |ga|. See |mbyte-combining|.
+
+    NOTE: the rexexp engine still has a hard-coded limit of considering
+    6 composing chars only.
+
   *'maxmem'* Nvim delegates memory-management to the OS.
   *'maxmemtot'* Nvim delegates memory-management to the OS.
   printoptions
diff --git a/runtime/lua/vim/_meta/options.lua b/runtime/lua/vim/_meta/options.lua
index 19ae786177..be4a4dd49c 100644
--- a/runtime/lua/vim/_meta/options.lua
+++ b/runtime/lua/vim/_meta/options.lua
@@ -2576,7 +2576,7 @@ vim.go.fp = vim.go.formatprg
 --- security reasons.
 ---
 --- @type boolean
-vim.o.fsync = false
+vim.o.fsync = true
 vim.o.fs = vim.o.fsync
 vim.go.fsync = vim.o.fsync
 vim.go.fs = vim.go.fsync