feat: try to recover from missing tempdir #22573

Problem:
If vim_tempdir mysteriously goes missing (typically by "antivirus" on
Windows), any plugins using tempname() will be broken for the rest of
the session. #1432 #9833 https://groups.google.com/g/vim_use/c/ef55jNm5czI
Steps:
    mkdir foo
    TMPDIR=./foo nvim
    :echo tempname()
    !rm -r foo
    :echo tempname()
    tempname() still uses the foo path even though it was deleted.

Solution:
- Don't assume that vim_tempdir exists.
- If it goes missing once, retry vim_mktempdir and log (silently) an error.
- If it goes missing again, retry vim_mktempdir and show an error.

Rejected in Vim for performance reasons:
  https://groups.google.com/g/vim_use/c/qgRob9SWDv8/m/FAOFVVcDTv0J
  https://groups.google.com/g/vim_dev/c/cogp-Vye4oo/m/d_SVFXBbnnoJ
But, logging shows that `vim_gettempdir` is not called frequently.

Fixes #1432
Fixes #9833
Fixes #11250
Related: stdpath("run") f50135a32e11c535e1dc3a8e9460c5b4e640ee86

6 files changed, 90 insertions, 29 deletions

diff --git a/runtime/doc/builtin.txt b/runtime/doc/builtin.txt
index b0b7809e8c..5e918a19f4 100644
--- a/runtime/doc/builtin.txt
+++ b/runtime/doc/builtin.txt
@@ -8688,13 +8688,11 @@ taglist({expr} [, {filename}])				*taglist()*
 			GetTagpattern()->taglist()
 
 tempname()					*tempname()* *temp-file-name*
-		The result is a String, which is the name of a file that
-		doesn't exist.  It can be used for a temporary file.  Example: >
+		Generates a (non-existent) filename located in the Nvim root
+		|tempdir|. Scripts can use the filename as a temporary file.
+		Example: >
 			:let tmpfile = tempname()
 			:exe "redir > " .. tmpfile
-<		For Unix, the file will be in a private directory |tempfile|.
-		For MS-Windows forward slashes are used when the 'shellslash'
-		option is set or when 'shellcmdflag' starts with '-'.
 
 termopen({cmd} [, {opts}])			*termopen()*
 		Spawns {cmd} in a new pseudo-terminal session connected
diff --git a/runtime/doc/change.txt b/runtime/doc/change.txt
index 2cebd8abdd..3706612d52 100644
--- a/runtime/doc/change.txt
+++ b/runtime/doc/change.txt
@@ -576,18 +576,29 @@ with ".".  Vim does not recognize a comment (starting with '"') after the
 {Visual}=		Filter the highlighted lines like with ={motion}.
 
 
-						*tempfile* *setuid*
-Vim uses temporary files for filtering, generating diffs and also for
-tempname().  For Unix, the file will be in a private directory (only
-accessible by the current user) to avoid security problems (e.g., a symlink
-attack or other people reading your file).  When Vim exits the directory and
-all files in it are deleted.  When Vim has the setuid bit set this may cause
-problems, the temp file is owned by the setuid user but the filter command
-probably runs as the original user.
-Directory for temporary files is created in the first possible directory of:
+						*tempdir* *tempfile* *setuid*
+Nvim uses temporary files for filtering and generating diffs. Plugins also
+commonly use |tempname()| for their own purposes. On the first request for
+a temporary file, Nvim creates a common directory (the "Nvim tempdir"), to
+serve as storage for all temporary files (including `stdpath("run")` files
+|$XDG_RUNTIME_DIR|) in the current session.
+
+The Nvim tempdir is created in the first available system tempdir:
 	Unix:    $TMPDIR, /tmp, current-dir, $HOME.
 	Windows: $TMPDIR, $TMP, $TEMP, $USERPROFILE, current-dir.
 
+On unix the tempdir is created with permissions 0700 (only accessible by the
+current user) to avoid security problems (e.g. symlink attacks).  On exit,
+Nvim deletes the tempdir and its contents.
+							*E5431*
+If you see an error or |log| message like: >
+	E5431: tempdir disappeared (2 times)
+this means an external process on your system deleted the Nvim tempdir.
+Typically this is caused by "antivirus" or a misconfigured cleanup service.
+
+If Nvim has the setuid bit set this may cause problems: the temp file
+is owned by the setuid user but the filter command probably runs as the
+original user.
 
 
 4.2 Substitute						*:substitute*
diff --git a/runtime/doc/starting.txt b/runtime/doc/starting.txt
index 8c9ec7df9c..801ea77b7f 100644
--- a/runtime/doc/starting.txt
+++ b/runtime/doc/starting.txt
@@ -1397,7 +1397,7 @@ Note: Similarly to the $XDG environment variables, when
 `$XDG_CONFIG_HOME/nvim` is mentionned, it should be understood as
 `$XDG_CONFIG_HOME/$NVIM_APPNAME`.
 
-LOG FILE					*$NVIM_LOG_FILE* *E5430*
+LOG FILE					*log* *$NVIM_LOG_FILE* *E5430*
 Besides 'debug' and 'verbose', Nvim keeps a general log file for internal
 debugging, plugins and RPC clients. >
 	:echo $NVIM_LOG_FILE
diff --git a/runtime/doc/vim_diff.txt b/runtime/doc/vim_diff.txt
index 2305f101e2..21736059c4 100644
--- a/runtime/doc/vim_diff.txt
+++ b/runtime/doc/vim_diff.txt
@@ -237,6 +237,7 @@ Functions:
   |stdpath()|
   |system()|, |systemlist()| can run {cmd} directly (without 'shell')
   |matchadd()| can be called before highlight group is defined
+  |tempname()| tries to recover if the Nvim |tempdir| disappears.
   |writefile()| with "p" flag creates parent directories.
 
 Highlight groups:
diff --git a/src/nvim/fileio.c b/src/nvim/fileio.c
index 6a13548027..115d34c3ea 100644
--- a/src/nvim/fileio.c
+++ b/src/nvim/fileio.c
@@ -5369,10 +5369,21 @@ void vim_deltempdir(void)
 /// Creates the directory on the first call.
 char *vim_gettempdir(void)
 {
-  if (vim_tempdir == NULL) {
+  static int notfound = 0;
+  bool exists = false;
+  if (vim_tempdir == NULL || !(exists = os_isdir(vim_tempdir))) {
+    if (vim_tempdir != NULL && !exists) {
+      notfound++;
+      if (notfound == 1) {
+        ELOG("tempdir disappeared (antivirus or broken cleanup job?): %s", vim_tempdir);
+      }
+      if (notfound > 1) {
+        msg_schedule_semsg("E5431: tempdir disappeared (%d times)", notfound);
+      }
+      XFREE_CLEAR(vim_tempdir);
+    }
     vim_mktempdir();
   }
-
   return vim_tempdir;
 }
 
diff --git a/test/functional/core/fileio_spec.lua b/test/functional/core/fileio_spec.lua
index 4e9891a4de..153b53dce2 100644
--- a/test/functional/core/fileio_spec.lua
+++ b/test/functional/core/fileio_spec.lua
@@ -15,6 +15,7 @@ local request = helpers.request
 local retry = helpers.retry
 local rmdir = helpers.rmdir
 local matches = helpers.matches
+local meths = helpers.meths
 local mkdir = helpers.mkdir
 local sleep = helpers.sleep
 local read_file = helpers.read_file
@@ -261,13 +262,13 @@ end)
 describe('tmpdir', function()
   local tmproot_pat = [=[.*[/\\]nvim%.[^/\\]+]=]
   local testlog = 'Xtest_tmpdir_log'
-  local faketmp
+  local os_tmpdir
 
   before_each(function()
     -- Fake /tmp dir so that we can mess it up.
-    faketmp = tmpname()
-    os.remove(faketmp)
-    mkdir(faketmp)
+    os_tmpdir = tmpname()
+    os.remove(os_tmpdir)
+    mkdir(os_tmpdir)
   end)
 
   after_each(function()
@@ -275,16 +276,21 @@ describe('tmpdir', function()
     os.remove(testlog)
   end)
 
-  it('failure modes', function()
-    clear({ env={ NVIM_LOG_FILE=testlog, TMPDIR=faketmp, } })
-    assert_nolog('tempdir is not a directory', testlog)
-    assert_nolog('tempdir has invalid permissions', testlog)
-
+  local function get_tmproot()
     -- Tempfiles typically look like: "…/nvim.<user>/xxx/0".
     --  - "…/nvim.<user>/xxx/" is the per-process tmpdir, not shared with other Nvims.
     --  - "…/nvim.<user>/" is the tmpdir root, shared by all Nvims (normally).
     local tmproot = (funcs.tempname()):match(tmproot_pat)
     ok(tmproot:len() > 4, 'tmproot like "nvim.foo"', tmproot)
+    return tmproot
+  end
+
+  it('failure modes', function()
+    clear({ env={ NVIM_LOG_FILE=testlog, TMPDIR=os_tmpdir, } })
+    assert_nolog('tempdir is not a directory', testlog)
+    assert_nolog('tempdir has invalid permissions', testlog)
+
+    local tmproot = get_tmproot()
 
     -- Test how Nvim handles invalid tmpdir root (by hostile users or accidents).
     --
@@ -292,7 +298,7 @@ describe('tmpdir', function()
     expect_exit(command, ':qall!')
     rmdir(tmproot)
     write_file(tmproot, '')  -- Not a directory, vim_mktempdir() should skip it.
-    clear({ env={ NVIM_LOG_FILE=testlog, TMPDIR=faketmp, } })
+    clear({ env={ NVIM_LOG_FILE=testlog, TMPDIR=os_tmpdir, } })
     matches(tmproot_pat, funcs.stdpath('run'))  -- Tickle vim_mktempdir().
     -- Assert that broken tmpdir root was handled.
     assert_log('tempdir root not a directory', testlog, 100)
@@ -303,18 +309,52 @@ describe('tmpdir', function()
     os.remove(tmproot)
     mkdir(tmproot)
     funcs.setfperm(tmproot, 'rwxr--r--')  -- Invalid permissions, vim_mktempdir() should skip it.
-    clear({ env={ NVIM_LOG_FILE=testlog, TMPDIR=faketmp, } })
+    clear({ env={ NVIM_LOG_FILE=testlog, TMPDIR=os_tmpdir, } })
     matches(tmproot_pat, funcs.stdpath('run'))  -- Tickle vim_mktempdir().
     -- Assert that broken tmpdir root was handled.
     assert_log('tempdir root has invalid permissions', testlog, 100)
   end)
 
   it('too long', function()
-    local bigname = ('%s/%s'):format(faketmp, ('x'):rep(666))
+    local bigname = ('%s/%s'):format(os_tmpdir, ('x'):rep(666))
     mkdir(bigname)
     clear({ env={ NVIM_LOG_FILE=testlog, TMPDIR=bigname, } })
     matches(tmproot_pat, funcs.stdpath('run'))  -- Tickle vim_mktempdir().
     local len = (funcs.tempname()):len()
     ok(len > 4 and len < 256, '4 < len < 256', tostring(len))
   end)
+
+  it('disappeared #1432', function()
+    clear({ env={ NVIM_LOG_FILE=testlog, TMPDIR=os_tmpdir, } })
+    assert_nolog('tempdir disappeared', testlog)
+
+    local function rm_tmpdir()
+      local tmpname1 = funcs.tempname()
+      local tmpdir1 = funcs.fnamemodify(tmpname1, ':h')
+      eq(funcs.stdpath('run'), tmpdir1)
+
+      rmdir(tmpdir1)
+      retry(nil, 1000, function()
+        eq(0, funcs.isdirectory(tmpdir1))
+      end)
+      local tmpname2 = funcs.tempname()
+      local tmpdir2 = funcs.fnamemodify(tmpname2, ':h')
+      neq(tmpdir1, tmpdir2)
+    end
+
+    -- Your antivirus hates you...
+    rm_tmpdir()
+    assert_log('tempdir disappeared', testlog, 100)
+    funcs.tempname()
+    funcs.tempname()
+    funcs.tempname()
+    eq('', meths.get_vvar('errmsg'))
+    rm_tmpdir()
+    funcs.tempname()
+    funcs.tempname()
+    funcs.tempname()
+    eq('E5431: tempdir disappeared (2 times)', meths.get_vvar('errmsg'))
+    rm_tmpdir()
+    eq('E5431: tempdir disappeared (3 times)', meths.get_vvar('errmsg'))
+  end)
 end)