fix(api): disallow some more functions during textlock

Problem: nvim_buf_set_text(), nvim_open_term() and termopen() all change buffer
text, which is forbidden during textlock. Additionally, nvim_open_term() and
termopen() may be used to convert the cmdwin buffer into a terminal buffer,
which is weird.

Solution: Allow nvim_buf_set_text() and nvim_open_term() in the cmdwin, but
disallow nvim_open_term() from converting the cmdwin buffer into a terminal
buffer. termopen() is not allowed in the cmdwin (as it always operates on
curbuf), so just check text_locked().

Also happens to improve the error in #21055: nvim_buf_set_text() was callable
during textlock, but happened to check textlock indirectly via u_save();
however, this caused the error to be overwritten by an unhelpful "Failed to
save undo information" message when msg_list == NULL (e.g: an `<expr>` mapping
invoked outside of do_cmdline()).

1 files changed, 6 insertions, 0 deletions

diff --git a/src/nvim/api/vim.c b/src/nvim/api/vim.c
index eacc833c58..8c40e5ccd7 100644
--- a/src/nvim/api/vim.c
+++ b/src/nvim/api/vim.c
@@ -951,12 +951,18 @@ fail:
 /// @return Channel id, or 0 on error
 Integer nvim_open_term(Buffer buffer, DictionaryOf(LuaRef) opts, Error *err)
   FUNC_API_SINCE(7)
+  FUNC_API_TEXTLOCK_ALLOW_CMDWIN
 {
   buf_T *buf = find_buffer_by_handle(buffer, err);
   if (!buf) {
     return 0;
   }
 
+  if (cmdwin_type != 0 && buf == curbuf) {
+    api_set_error(err, kErrorTypeException, "%s", _(e_cmdwin));
+    return 0;
+  }
+
   LuaRef cb = LUA_NOREF;
   for (size_t i = 0; i < opts.size; i++) {
     String k = opts.items[i].key;