feat(server): set $NVIM, unset $NVIM_LISTEN_ADDRESS #11009

PROBLEM
------------------------------------------------------------------------
$NVIM_LISTEN_ADDRESS has conflicting purposes as both a parameter ("the
current process should listen on this address") and a descriptor ("the
current process is a child of this address").

This contradiction means the presence of NVIM_LISTEN_ADDRESS is
ambiguous, so child Nvim always tries to listen on its _parent's_
socket. This is the cause of lots of  "Failed to start server" spam in
our test/CI logs:

    WARN  2022-04-30… server_start:154: Failed to start server: address already in use: \\.\pipe\nvim-4480-0
    WARN  2022-04-30… server_start:154: Failed to start server: address already in use: \\.\pipe\nvim-2168-0

SOLUTION
------------------------------------------------------------------------

1. Set $NVIM to the parent v:servername, *only* in child processes.
   - Now the correct way to detect a "parent" Nvim is to check for $NVIM.
2. Do NOT set $NVIM_LISTEN_ADDRESS in child processes.
3. On startup if $NVIM_LISTEN_ADDRESS exists, unset it immediately after
   server init.
4. Open a channel to parent automatically, expose it as v:parent.

Fixes #3118
Fixes #6764
Fixes #9336
Ref https://github.com/neovim/neovim/pull/8247#issuecomment-380275696
Ref #8696

5 files changed, 56 insertions, 177 deletions

diff --git a/runtime/doc/api.txt b/runtime/doc/api.txt
index ed3a838b6d..ed13d7f8c2 100644
--- a/runtime/doc/api.txt
+++ b/runtime/doc/api.txt
@@ -59,7 +59,7 @@ Nvim instance:
     # trailing '&' which is required since Nvim won't process events while
     # running a blocking command):
     #
-    #	:!./hello.rb &
+    #   :!./hello.rb &
     #
     # Or from another shell by setting NVIM_LISTEN_ADDRESS:
     # $ NVIM_LISTEN_ADDRESS=[address] ./hello.rb
diff --git a/runtime/doc/builtin.txt b/runtime/doc/builtin.txt
index 3e75914743..a909fd0d6b 100644
--- a/runtime/doc/builtin.txt
+++ b/runtime/doc/builtin.txt
@@ -158,7 +158,6 @@ foldclosedend({lnum})		Number	last line of fold at {lnum} if closed
 foldlevel({lnum})		Number	fold level at {lnum}
 foldtext()			String	line displayed for closed fold
 foldtextresult({lnum})		String	text for closed fold at {lnum}
-foreground()			Number	bring the Vim window to the foreground
 fullcommand({name})		String	get full command from {name}
 funcref({name} [, {arglist}] [, {dict}])
 				Funcref	reference to function {name}
@@ -352,16 +351,6 @@ reg_recording()			String	get the recording register name
 reltime([{start} [, {end}]])	List	get time value
 reltimefloat({time})		Float	turn the time value into a Float
 reltimestr({time})		String	turn time value into a String
-remote_expr({server}, {string} [, {idvar} [, {timeout}]])
-				String	send expression
-remote_foreground({server})	Number	bring Vim server to the foreground
-remote_peek({serverid} [, {retvar}])
-				Number	check for reply string
-remote_read({serverid} [, {timeout}])
-				String	read reply string
-remote_send({server}, {string} [, {idvar}])
-				String	send key sequence
-remote_startserver({name})	none	become server {name}
 remove({list}, {idx} [, {end}])	any/List
 					remove items {idx}-{end} from {list}
 remove({blob}, {idx} [, {end}])	Number/Blob
@@ -395,8 +384,6 @@ searchpairpos({start}, {middle}, {end} [, {flags} [, {skip} [...]]])
 				List	search for other end of start/end pair
 searchpos({pattern} [, {flags} [, {stopline} [, {timeout} [, {skip}]]]])
 				List	search for {pattern}
-server2client({clientid}, {string})
-				Number	send reply string
 serverlist()			String	get a list of available servers
 setbufline({expr}, {lnum}, {text})
 				Number	set line {lnum} to {text} in buffer
@@ -1974,7 +1961,7 @@ expand({string} [, {nosuf} [, {list}]])				*expand()*
 			<cword>		word under the cursor
 			<cWORD>		WORD under the cursor
 			<client>	the {clientid} of the last received
-					message |server2client()|
+					message
 		Modifiers:
 			:p		expand to full path
 			:h		head (last path component removed)
@@ -2409,14 +2396,6 @@ foldtextresult({lnum})					*foldtextresult()*
 		Can also be used as a |method|: >
 			GetLnum()->foldtextresult()
 <
-							*foreground()*
-foreground()	Move the Vim window to the foreground.  Useful when sent from
-		a client to a Vim server. |remote_send()|
-		On Win32 systems this might not work, the OS does not always
-		allow a window to bring itself to the foreground.  Use
-		|remote_foreground()| instead.
-		{only in the Win32 GUI and console version}
-
 fullcommand({name})						*fullcommand()*
 		Get the full command name from a short abbreviated command
 		name; see |20.2| for details on command abbreviations.
@@ -4280,6 +4259,15 @@ jobstart({cmd} [, {opts}])				*jobstart()*
 		    by CommandLineToArgvW https://msdn.microsoft.com/bb776391
 		    unless cmd[0] is some form of "cmd.exe".
 
+							*jobstart-env*
+		The job environment is initialized as follows:
+		  $NVIM                is set to |v:servername| of the parent Nvim
+		  $NVIM_LISTEN_ADDRESS is unset
+		  $NVIM_LOG_FILE       is unset
+		  $VIM                 is unset
+		  $VIMRUNTIME          is unset
+		You can set these with the `env` option.
+
 							*jobstart-options*
 		{opts} is a dictionary with these keys:
 		  clear_env:  (boolean) `env` defines the job environment
@@ -4290,8 +4278,8 @@ jobstart({cmd} [, {opts}])				*jobstart()*
 			      killed when Nvim exits. If the process exits
 			      before Nvim, `on_exit` will be invoked.
 		  env:	      (dict) Map of environment variable name:value
-			      pairs extending (or replacing if |clear_env|)
-			      the current environment.
+			      pairs extending (or replacing with |clear_env|)
+			      the current environment. |jobstart-env|
 		  height:     (number) Height of the `pty` terminal.
 		  |on_exit|:    (function) Callback invoked when the job exits.
 		  |on_stdout|:  (function) Callback invoked when the job emits
@@ -4305,8 +4293,8 @@ jobstart({cmd} [, {opts}])				*jobstart()*
 			      platforms, this option is silently ignored.)
 		  pty:	      (boolean) Connect the job to a new pseudo
 			      terminal, and its streams to the master file
-			      descriptor. Then  `on_stderr` is ignored,
-			      `on_stdout` receives all output.
+			      descriptor. `on_stdout` receives all output,
+			      `on_stderr` is ignored. |terminal-start|
 		  rpc:	      (boolean) Use |msgpack-rpc| to communicate with
 			      the job over stdio. Then `on_stdout` is ignored,
 			      but `on_stderr` can still be used.
@@ -5242,9 +5230,8 @@ mode([expr])	Return a string that indicates the current mode.
 		   !	    Shell or external command is executing
 		   t	    Terminal mode: keys go to the job
 
-		This is useful in the 'statusline' option or when used
-		with |remote_expr()| In most other places it always returns
-		"c" or "n".
+		This is useful in the 'statusline' option or RPC calls. In
+		most other places it always returns "c" or "n".
 		Note that in the future more modes and more specific modes may
 		be added. It's better not to compare the whole string but only
 		the leading character(s).
@@ -5964,107 +5951,6 @@ reltimestr({time})				*reltimestr()*
 		Can also be used as a |method|: >
 			reltime(start)->reltimestr()
 <
-							*remote_expr()* *E449*
-remote_expr({server}, {string} [, {idvar} [, {timeout}]])
-		Send the {string} to {server}.  The {server} argument is a
-		string, also see |{server}|.
-
-		The string is sent as an expression and the result is returned
-		after evaluation.  The result must be a String or a |List|.  A
-		|List| is turned into a String by joining the items with a
-		line break in between (not at the end), like with join(expr,
-		"\n").
-
-		If {idvar} is present and not empty, it is taken as the name
-		of a variable and a {serverid} for later use with
-		|remote_read()| is stored there.
-
-		If {timeout} is given the read times out after this many
-		seconds.  Otherwise a timeout of 600 seconds is used.
-
-		See also |clientserver| |RemoteReply|.
-		This function is not available in the |sandbox|.
-		Note: Any errors will cause a local error message to be issued
-		and the result will be the empty string.
-
-		Variables will be evaluated in the global namespace,
-		independent of a function currently being active.  Except
-		when in debug mode, then local function variables and
-		arguments can be evaluated.
-
-		Examples: >
-			:echo remote_expr("gvim", "2+2")
-			:echo remote_expr("gvim1", "b:current_syntax")
-<
-
-remote_foreground({server})				*remote_foreground()*
-		Move the Vim server with the name {server} to the foreground.
-		The {server} argument is a string, also see |{server}|.
-		This works like: >
-			remote_expr({server}, "foreground()")
-<		Except that on Win32 systems the client does the work, to work
-		around the problem that the OS doesn't always allow the server
-		to bring itself to the foreground.
-		Note: This does not restore the window if it was minimized,
-		like foreground() does.
-		This function is not available in the |sandbox|.
-		{only in the Win32 GUI and the Win32 console version}
-
-
-remote_peek({serverid} [, {retvar}])		*remote_peek()*
-		Returns a positive number if there are available strings
-		from {serverid}.  Copies any reply string into the variable
-		{retvar} if specified.  {retvar} must be a string with the
-		name of a variable.
-		Returns zero if none are available.
-		Returns -1 if something is wrong.
-		See also |clientserver|.
-		This function is not available in the |sandbox|.
-		Examples: >
-			:let repl = ""
-			:echo "PEEK: " .. remote_peek(id, "repl") .. ": " .. repl
-
-remote_read({serverid}, [{timeout}])			*remote_read()*
-		Return the oldest available reply from {serverid} and consume
-		it.  Unless a {timeout} in seconds is given, it blocks until a
-		reply is available.
-		See also |clientserver|.
-		This function is not available in the |sandbox|.
-		Example: >
-			:echo remote_read(id)
-<
-							*remote_send()* *E241*
-remote_send({server}, {string} [, {idvar}])
-		Send the {string} to {server}.  The {server} argument is a
-		string, also see |{server}|.
-
-		The string is sent as input keys and the function returns
-		immediately.  At the Vim server the keys are not mapped
-		|:map|.
-
-		If {idvar} is present, it is taken as the name of a variable
-		and a {serverid} for later use with remote_read() is stored
-		there.
-
-		See also |clientserver| |RemoteReply|.
-		This function is not available in the |sandbox|.
-
-		Note: Any errors will be reported in the server and may mess
-		up the display.
-		Examples: >
-		:echo remote_send("gvim", ":DropAndReply " .. file, "serverid") ..
-		 \ remote_read(serverid)
-
-		:autocmd NONE RemoteReply *
-		 \ echo remote_read(expand("<amatch>"))
-		:echo remote_send("gvim", ":sleep 10 | echo " ..
-		 \ 'server2client(expand("<client>"), "HELLO")<CR>')
-<
-					*remote_startserver()* *E941* *E942*
-remote_startserver({name})
-		Become the server {name}.  This fails if already running as a
-		server, when |v:servername| is not empty.
-
 remove({list}, {idx} [, {end}])				*remove()*
 		Without {end}: Remove the item at {idx} from |List| {list} and
 		return the item.
@@ -6648,21 +6534,6 @@ searchpos({pattern} [, {flags} [, {stopline} [, {timeout} [, {skip}]]]])
 		Can also be used as a |method|: >
 			GetPattern()->searchpos()
 
-server2client({clientid}, {string})			*server2client()*
-		Send a reply string to {clientid}.  The most recent {clientid}
-		that sent a string can be retrieved with expand("<client>").
-		Note:
-		Returns zero for success, -1 for failure.
-		This id has to be stored before the next command can be
-		received.  I.e. before returning from the received command and
-		before calling any commands that waits for input.
-		See also |clientserver|.
-		Example: >
-			:echo server2client(expand("<client>"), "HELLO")
-
-<		Can also be used as a |method|: >
-			GetClientId()->server2client(string)
-<
 serverlist()						*serverlist()*
 		Returns a list of server addresses, or empty if all servers
 		were stopped. |serverstart()| |serverstop()|
@@ -6672,7 +6543,9 @@ serverlist()						*serverlist()*
 serverstart([{address}])				*serverstart()*
 		Opens a socket or named pipe at {address} and listens for
 		|RPC| messages. Clients can send |API| commands to the address
-		to control Nvim. Returns the address string.
+		to control Nvim.
+
+		Returns the address string.
 
 		If {address} does not contain a colon ":" it is interpreted as
 		a named pipe or Unix domain socket path.
@@ -6694,14 +6567,11 @@ serverstart([{address}])				*serverstart()*
 		If no address is given, it is equivalent to: >
 			:call serverstart(tempname())
 
-< 		|$NVIM_LISTEN_ADDRESS| is set to {address} if not already set.
-
 serverstop({address})					*serverstop()*
 		Closes the pipe or socket at {address}.
 		Returns TRUE if {address} is valid, else FALSE.
-		If |$NVIM_LISTEN_ADDRESS| is stopped it is unset.
 		If |v:servername| is stopped it is set to the next available
-		address returned by |serverlist()|.
+		address in |serverlist()|.
 
 setbufline({buf}, {lnum}, {text})			*setbufline()*
 		Set line {lnum} to {text} in buffer {buf}.  This works like
@@ -8281,19 +8151,15 @@ tempname()					*tempname()* *temp-file-name*
 
 termopen({cmd} [, {opts}])			*termopen()*
 		Spawns {cmd} in a new pseudo-terminal session connected
-		to the current buffer.  {cmd} is the same as the one passed to
-		|jobstart()|.  This function fails if the current buffer is
-		modified (all buffer contents are destroyed).
-
-		The {opts} dict is similar to the one passed to |jobstart()|,
-		but the `pty`, `width`, `height`, and `TERM` fields are
-		ignored: `height`/`width` are taken from the current window
-		and `$TERM` is set to "xterm-256color".
+		to the current (unmodified) buffer. Parameters and behavior
+		are the same as |jobstart()| except "pty", "width", "height",
+		and "TERM" are ignored: "height" and "width" are taken from
+		the current window.
 		Returns the same values as |jobstart()|.
 
-		See |terminal| for more information.
-
-test_ functions are documented here: |test-functions-details|
+		Terminal environment is initialized as in ||jobstart-env|,
+		except $TERM is set to "xterm-256color". Full behavior is
+		described in |terminal|.
 
 tan({expr})						*tan()*
 		Return the tangent of {expr}, measured in radians, as a |Float|
diff --git a/runtime/doc/deprecated.txt b/runtime/doc/deprecated.txt
index f24bf06feb..13644cf208 100644
--- a/runtime/doc/deprecated.txt
+++ b/runtime/doc/deprecated.txt
@@ -6,9 +6,8 @@
 
 Nvim                                                             *deprecated*
 
-The items listed below are "deprecated". This means they will be removed in
-the future. They should not be used in new scripts, and old scripts should be
-updated.
+The items listed below are deprecated: they will be removed in the future.
+They should not be used in new scripts, and old scripts should be updated.
 
 ==============================================================================
 
@@ -25,8 +24,12 @@ Commands ~
 *:wviminfo*		Deprecated alias to |:wshada| command.
 
 Environment Variables ~
-*$NVIM_LISTEN_ADDRESS*	Deprecated in favor of |--listen|.  If both are given,
-			$NVIM_LISTEN_ADDRESS is ignored.
+*$NVIM_LISTEN_ADDRESS*	$NVIM_LISTEN_ADDRESS is a deprecated way to set the
+			|--listen| address of Nvim, and also had a conflicting
+			purpose as a way to detect a parent Nvim (use |$NVIM|
+			for that). It is unset by |terminal| and |jobstart()|
+			(unless explicitly given by the "env" option).
+			Ignored if --listen is given.
 
 Events ~
 *BufCreate*		Use |BufAdd| instead.
diff --git a/runtime/doc/eval.txt b/runtime/doc/eval.txt
index c6319c717a..3fd0d96f21 100644
--- a/runtime/doc/eval.txt
+++ b/runtime/doc/eval.txt
@@ -2138,9 +2138,19 @@ v:scrollstart	String describing the script or function that caused the
 					*v:servername* *servername-variable*
 v:servername	Primary listen-address of the current Nvim instance, the first
 		item returned by |serverlist()|.  Can be set by |--listen| or
-		|$NVIM_LISTEN_ADDRESS| at startup. |serverstart()| |serverstop()|
+		|$NVIM_LISTEN_ADDRESS| (deprecated) at startup.
+		See also |serverstart()| |serverstop()|.
 		Read-only.
 
+								       *$NVIM*
+		$NVIM is set by |terminal| and |jobstart()|, and is thus
+		a hint that the current environment is a subprocess of Nvim.
+		Example: >
+		    if $NVIM
+		      echo nvim_get_chan_info(v:parent)
+		    endif
+
+<		Note the contents of $NVIM may change in the future.
 
 v:searchforward			*v:searchforward* *searchforward-variable*
 		Search direction:  1 after a forward search, 0 after a
diff --git a/runtime/doc/nvim_terminal_emulator.txt b/runtime/doc/nvim_terminal_emulator.txt
index 0e2b048541..49e29111c6 100644
--- a/runtime/doc/nvim_terminal_emulator.txt
+++ b/runtime/doc/nvim_terminal_emulator.txt
@@ -25,23 +25,23 @@ Start						*terminal-start*
 
 There are several ways to create a terminal buffer:
 
-- Invoke the |:terminal| command.
-- Call the |termopen()| function.
-- Edit a file with a name matching `term://(.{-}//(\d+:)?)?\zs.*`.
-  For example:
->
+- Run the |:terminal| command.
+- Call the |nvim_open_term()| or |termopen()| function.
+- Edit a "term://" buffer. Examples: >
     :edit term://bash
     :vsplit term://top
-<
-    Note: The "term://" pattern is handled by a BufReadCmd handler, so the
-    |autocmd-nested| modifier is required to use it in an autocmd. >
+
+<    Note: To open a "term://" buffer from an autocmd, the |autocmd-nested|
+    modifier is required. >
         autocmd VimEnter * ++nested split term://sh
-<    This is only mentioned for reference; use |:terminal| instead.
+<    (This is only mentioned for reference; use |:terminal| instead.)
 
 When the terminal starts, the buffer contents are updated and the buffer is
 named in the form of `term://{cwd}//{pid}:{cmd}`. This naming scheme is used
 by |:mksession| to restore a terminal buffer (by restarting the {cmd}).
 
+The terminal environment is initialized as in |jobstart-env|.
+
 ==============================================================================
 Input						*terminal-input*