doc: eliminate msgpack_rpc.txt [ci skip]

- Migrate msgpack_rpc.txt into api.txt, develop.txt.
- fix #10740: Remove warning about "avoid hardcoding the type codes".

6 files changed, 180 insertions, 265 deletions

diff --git a/runtime/doc/api.txt b/runtime/doc/api.txt
index c372bb401e..3b641760f7 100644
--- a/runtime/doc/api.txt
+++ b/runtime/doc/api.txt
@@ -14,26 +14,113 @@ Applications can also embed libnvim to work with the C API directly.
                                       Type |gO| to see the table of contents.
 
 ==============================================================================
+API Usage						*api-rpc* *RPC* *rpc*
+
+							*msgpack-rpc*
+RPC is the typical way to control Nvim programmatically.  Nvim implements the
+MessagePack-RPC protocol:
+  https://github.com/msgpack/msgpack/blob/0b8f5ac/spec.md
+
+Many clients use the API: user interfaces (GUIs), remote plugins, scripts like
+"nvr" (https://github.com/mhinz/neovim-remote).  Even Nvim itself can control
+other Nvim instances.  API clients can:
+
+  - Call any API function
+  - Listen for events
+  - Receive remote calls from Nvim
+
+The RPC API is like a more powerful version of Vim's "clientserver" feature.
+
+CONNECTING						*rpc-connecting*
+
+See |channel-intro| for various ways to open a channel. Channel-opening
+functions take an `rpc` key in the options dictionary. RPC channels can also
+be opened by other processes connecting to TCP/IP sockets or named pipes
+listened to by Nvim.
+
+Nvim creates a default RPC socket at |startup|, given by |v:servername|. To
+start with a TCP/IP socket instead, use |--listen| with a TCP-style address: >
+    nvim --listen 127.0.0.1:6666
+More endpoints can be started with |serverstart()|.
+
+Note that localhost TCP sockets are generally less secure than named pipes,
+and can lead to vunerabilities like remote code execution.
+
+Connecting to the socket is the easiest way a programmer can test the API,
+which can be done through any msgpack-rpc client library or full-featured
+|api-client|. Here's a Ruby script that prints "hello world!" in the current
+Nvim instance:
+>
+    #!/usr/bin/env ruby
+    # Requires msgpack-rpc: gem install msgpack-rpc
+    #
+    # To run this script, execute it from a running Nvim instance (notice the
+    # trailing '&' which is required since Nvim won't process events while
+    # running a blocking command):
+    #
+    #	:!./hello.rb &
+    #
+    # Or from another shell by setting NVIM_LISTEN_ADDRESS:
+    # $ NVIM_LISTEN_ADDRESS=[address] ./hello.rb
+
+    require 'msgpack/rpc'
+    require 'msgpack/rpc/transport/unix'
+
+    nvim = MessagePack::RPC::Client.new(MessagePack::RPC::UNIXTransport.new, ENV['NVIM_LISTEN_ADDRESS'])
+    result = nvim.call(:nvim_command, 'echo "hello world!"')
+<
+A better way is to use the Python REPL with the `neovim` package, where API
+functions can be called interactively:
+>
+    >>> from pynvim import attach
+    >>> nvim = attach('socket', path='[address]')
+    >>> nvim.command('echo "hello world!"')
+<
+You can also embed Nvim via |jobstart()|, and communicate using |rpcrequest()|
+and |rpcnotify()|:
+>
+    let nvim = jobstart(['nvim', '--embed'], {'rpc': v:true})
+    echo rpcrequest(nvim, 'nvim_eval', '"Hello " . "world!"')
+    call jobstop(nvim)
+
+==============================================================================
 API Definitions						*api-definitions*
 
 							*api-types*
 The Nvim C API defines custom types for all function parameters. Some are just
 typedefs around C99 standard types, others are Nvim-defined data structures.
 
-  Boolean				-> bool
-  Integer (signed 64-bit integer)	-> int64_t
-  Float (IEEE 754 double precision)	-> double
-  String				-> {char* data, size_t size} struct
+Basic types ~
+
+  API Type				C type
+  ------------------------------------------------------------------------
+  Nil
+  Boolean				bool
+  Integer (signed 64-bit integer)	int64_t
+  Float (IEEE 754 double precision)	double
+  String				{char* data, size_t size} struct
   Array
-  Dictionary
+  Dictionary (msgpack: map)
   Object
 
-The following handle types are defined as integer typedefs, but are
-discriminated as separate types in an Object:
+  Note: empty Array is accepted as a valid argument for Dictionary parameter.
+
+Special types (msgpack EXT) ~
+
+  These are integer typedefs discriminated as separate Object subtypes. They
+  can be treated as opaque integers, but are mutually incompatible: Buffer may
+  be passed as an integer but not as Window or Tabpage.
+
+  The EXT object data is the (integer) object handle. The EXT type codes given
+  in the |api-metadata| `types` key are stable: they will not change and are
+  thus forward-compatible.
+
+  EXT Type	C type					Data
+  ------------------------------------------------------------------------
+  Buffer	enum value kObjectTypeBuffer		|bufnr()|
+  Window	enum value kObjectTypeWindow		|window-ID|
+  Tabpage	enum value kObjectTypeTabpage		internal handle
 
-  Buffer				-> enum value kObjectTypeBuffer
-  Window				-> enum value kObjectTypeWindow
-  Tabpage				-> enum value kObjectTypeTabpage
 
 							*api-indexing*
 Most of the API uses 0-based indices, and ranges are end-exclusive. For the
@@ -47,24 +134,61 @@ lines, 0-based columns):
     |nvim_win_set_cursor()|
 
 ==============================================================================
-API metadata							 *api-metadata*
+API metadata						*api-metadata*
 
-Nvim exposes API metadata as a Dictionary. Some items are described below:
+The Nvim C API is automatically exposed to RPC by the build system, which
+parses headers in src/nvim/api/* and generates dispatch-functions mapping RPC
+API method names to public C API functions, converting/validating arguments
+and return values.
+
+Nvim exposes its API metadata as a Dictionary with these items:
 
 version			Nvim version, API level/compatibility
 version.api_level	API version integer *api-level*
 version.api_compatible	API is backwards-compatible with this level
 version.api_prerelease	Declares the API as unstable/unreleased >
 			(version.api_prerelease && fn.since == version.api_level)
-functions		API function signatures
-ui_events		UI event signatures |ui|
+functions		API function signatures, containing |api-types| info
+                        describing the return value and parameters.
+ui_events		|UI| event signatures
 ui_options		Supported |ui-option|s
 {fn}.since		API level where function {fn} was introduced
 {fn}.deprecated_since	API level where function {fn} was deprecated
 types			Custom handle types defined by Nvim
 error_types		Possible error types returned by API functions
 
-External programs ("clients") can use the metadata to discover the |rpc-api|.
+About the `functions` map:
+
+  - Container types may be decorated with type/size constraints, e.g.
+    ArrayOf(Buffer) or ArrayOf(Integer, 2).
+  - Functions considered to be methods that operate on instances of Nvim
+    special types (msgpack EXT) have the "method=true" flag. The receiver type
+    is that of the first argument. Method names are prefixed with `nvim_` plus
+    a type name, e.g. `nvim_buf_get_lines` is the `get_lines` method of
+    a Buffer instance. |dev-api|
+  - Global functions have the "method=false" flag and are prefixed with just
+    `nvim_`, e.g. `nvim_get_buffers`.
+
+							*api-mapping*
+External programs (clients) can use the metadata to discover the API, using
+any of these approaches:
+
+  1. Connect to a running Nvim instance and call |nvim_get_api_info()| via
+     msgpack-rpc. This is best for clients written in dynamic languages which
+     can define functions at runtime.
+
+  2. Start Nvim with the |--api-info| option. Useful for clients written in
+     statically-compiled languages.
+
+  3. Use the |api_info()| Vimscript function.
+
+Example: To get a human-readable list of API functions: >
+    :new|put =map(filter(api_info().functions, '!has_key(v:val,''deprecated_since'')'), 'v:val.name')
+<
+Example: To get a formatted dump of the API using python (requires the
+"pyyaml" and "msgpack-python" modules): >
+    nvim --api-info | python -c 'import msgpack, sys, yaml; print yaml.dump(msgpack.unpackb(sys.stdin.read()))'
+<
 
 ==============================================================================
 API contract                                                     *api-contract*
diff --git a/runtime/doc/channel.txt b/runtime/doc/channel.txt
index f3fa3fa180..967f4b26f2 100644
--- a/runtime/doc/channel.txt
+++ b/runtime/doc/channel.txt
@@ -30,7 +30,7 @@ There are several ways to open a channel:
 
 Channels support multiple modes or protocols. In the most basic
 mode of operation, raw bytes are read and written to the channel.
-The |rpc| protocol, based on the msgpack-rpc standard, enables nvim and the
+The |RPC| protocol, based on the msgpack-rpc standard, enables nvim and the
 process at the other end to send remote calls and events to each other.
 The builtin |terminal-emulator| is also implemented on top of PTY channels.
 
diff --git a/runtime/doc/develop.txt b/runtime/doc/develop.txt
index 3262d9dec7..90c2e30771 100644
--- a/runtime/doc/develop.txt
+++ b/runtime/doc/develop.txt
@@ -185,11 +185,25 @@ Example:
 
 API-CLIENT						*dev-api-client*
 
+							*api-client*
+API clients wrap the Nvim |API| to provide idiomatic "SDKs" for their
+respective platforms (see |jargon|). You can build a new API client for your
+favorite platform or programming language.
+
+List of API clients:
+    https://github.com/neovim/neovim/wiki/Related-projects#api-clients
+
+							*pynvim*
+The Python client is the reference implementation for API clients.
+    https://github.com/neovim/pynvim
+
 Standard Features ~
 
+- API clients exist to hide msgpack-rpc details. The wrappers can be
+  automatically generated by reading the |api-metadata| from Nvim. |api-mapping|
 - Clients should call |nvim_set_client_info()| after connecting, so users and
-  plugins can detect the client by handling the |ChanInfo| event. This
-  avoids the need for special variables or other client hints.
+  plugins can detect the client by handling the |ChanInfo| event. This avoids
+  the need for special variables or other client hints.
 - Clients should handle |nvim_error_event| notifications, which will be sent
   if an async request to nvim was rejected or caused an error.
 
@@ -209,15 +223,29 @@ Examples of API-client package names:
         BAD:  python-client
         BAD:  neovim
 
-Implementation ~
-
-For C/C++ projects, consider libmpack instead of the msgpack.org library.
+API client implementation guidelines ~
+
+- Separate the transport layer from the rest of the library. |rpc-connecting|
+- Use a MessagePack library that implements at least version 5 of the
+  MessagePack spec, which supports the BIN and EXT types used by Nvim.
+- Use a single-threaded event loop library/pattern.
+- Use a fiber/coroutine library for the language being used for implementing
+  a client. These greatly simplify concurrency and allow the library to
+  expose a blocking API on top of a non-blocking event loop without the
+  complexity that comes with preemptive multitasking.
+- Don't assume anything about the order of responses to RPC requests.
+- Clients should expect requests, which must be handled immediately because
+  Nvim is blocked while waiting for the client response.
+- Clients should expect notifications, but these can be handled "ASAP" (rather
+  than immediately) because they won't block Nvim.
+- For C/C++ projects, consider libmpack instead of the msgpack.org library.
     https://github.com/libmpack/libmpack/
-libmpack is small (no dependencies, can inline into your C/C++ project) and
-efficient (no allocations). It also implements msgpack-RPC, the protocol
-required by Nvim.
+  libmpack is small (no dependencies, can inline into your C/C++ project) and
+  efficient (no allocations). It also implements msgpack-RPC, the protocol
+  required by Nvim.
     https://github.com/msgpack-rpc/msgpack-rpc
 
+
 EXTERNAL UI 						*dev-ui*
 
 External UIs should be aware of the |api-contract|. In particular, future
diff --git a/runtime/doc/msgpack_rpc.txt b/runtime/doc/msgpack_rpc.txt
index b24e706aa9..f5d42dfeb2 100644
--- a/runtime/doc/msgpack_rpc.txt
+++ b/runtime/doc/msgpack_rpc.txt
@@ -3,242 +3,5 @@
 		 NVIM REFERENCE MANUAL    by Thiago de Arruda
 
 
-RPC API for Nvim				     *RPC* *rpc* *msgpack-rpc*
 
-				      Type |gO| to see the table of contents.
-
-==============================================================================
-1. Introduction						            *rpc-intro*
-
-The primary way to control Nvim programmatically is the RPC API, which speaks
-MessagePack-RPC ("msgpack-rpc"), a messaging protocol that uses the
-MessagePack serialization format:
-  https://github.com/msgpack/msgpack/blob/0b8f5ac/spec.md
-
-All kinds of Nvim "clients" use the RPC API: user interfaces (GUIs), remote
-plugins, scripts like "nvr" (https://github.com/mhinz/neovim-remote), and even
-`nvim` itself can control other `nvim` instances. By connecting to the RPC API
-programs can:
-
-  - Call any API function
-  - Listen for events
-  - Receive remote calls from Nvim
-
-The RPC API is like a more powerful version of Vim's `clientserver` feature.
-
-==============================================================================
-2. API mapping							    *rpc-api*
-
-The Nvim C |API| is automatically exposed to the RPC API by the build system,
-which parses headers at src/nvim/api/*. A dispatch function is generated which
-matches RPC API method names with public API functions, converting/validating
-arguments and return values back to msgpack.
-
-Client libraries (|api-client|s) normally provide wrappers that hide
-msgpack-rpc details from application developers. The wrappers can be
-automatically generated by reading bundled API metadata from a compiled Nvim
-instance.
-
-There are three ways to obtain API metadata:
-
-  1. Connect to a running Nvim instance and call `nvim_get_api_info` via
-     msgpack-rpc. This is best for clients written in dynamic languages which
-     can define functions at runtime.
-
-  2. Start Nvim with the |--api-info| option. Useful for clients written in
-     statically-compiled languages.
-
-  3. Use the |api_info()| vimscript function.
-
-To get a human-readable list of API functions: >
-    :new|put =map(filter(api_info().functions, '!has_key(v:val,''deprecated_since'')'), 'v:val.name')
-<
-To get a formatted dump of the API using python (requires the `pyyaml` and
-`msgpack-python` packages): >
-    nvim --api-info | python -c 'import msgpack, sys, yaml; print yaml.dump(msgpack.unpackb(sys.stdin.read()))'
-<
-==============================================================================
-3. Connecting						      *rpc-connecting*
-
-See |channel-intro|, for various ways to open a channel. Most of the channel
-opening functions take an `rpc` key in the options dictionary, to enable RPC.
-
-Additionally, RPC channels can be opened by other processes connecting to
-TCP/IP sockets or named pipes listened to by nvim.
-
-Nvim creates a default RPC socket at |startup|, given by |v:servername|. To
-start with a TCP/IP socket instead, use |--listen| with a TCP-style address: >
-    nvim --listen 127.0.0.1:6666
-Additional sockets and named pipes can be started with |serverstart()|.
-
-Note that localhost TCP sockets are generally less secure than named pipes,
-and can lead to vunerabilities like remote code execution.
-
-Connecting to the socket is the easiest way a programmer can test the API,
-which can be done through any msgpack-rpc client library or full-featured
-|api-client|. Here's a Ruby script that prints 'hello world!' in the current
-Nvim instance:
->
-    #!/usr/bin/env ruby
-    # Requires msgpack-rpc: gem install msgpack-rpc
-    #
-    # To run this script, execute it from a running Nvim instance (notice the
-    # trailing '&' which is required since Nvim won't process events while
-    # running a blocking command):
-    #
-    #	:!./hello.rb &
-    #
-    # Or from another shell by setting NVIM_LISTEN_ADDRESS:
-    # $ NVIM_LISTEN_ADDRESS=[address] ./hello.rb
-
-    require 'msgpack/rpc'
-    require 'msgpack/rpc/transport/unix'
-
-    nvim = MessagePack::RPC::Client.new(MessagePack::RPC::UNIXTransport.new, ENV['NVIM_LISTEN_ADDRESS'])
-    result = nvim.call(:nvim_command, 'echo "hello world!"')
-<
-A better way is to use the Python REPL with the `neovim` package, where API
-functions can be called interactively:
->
-    >>> from neovim import attach
-    >>> nvim = attach('socket', path='[address]')
-    >>> nvim.command('echo "hello world!"')
-<
-You can also embed an Nvim instance via |jobstart()|, and communicate using
-|rpcrequest()| and |rpcnotify()|:
->
-    let nvim = jobstart(['nvim', '--embed'], {'rpc': v:true})
-    echo rpcrequest(nvim, 'nvim_eval', '"Hello " . "world!"')
-    call jobstop(nvim)
-<
-==============================================================================
-4. Implementing API clients			*rpc-api-client* *api-client*
-
-API clients wrap the Nvim API to provide idiomatic "SDKs" for their respective
-platforms (see |jargon|). You can build a new API client for your favorite
-platform or programming language.
-
-API clients are listed here:
-    https://github.com/neovim/neovim/wiki/Related-projects#api-clients
-
-                                                                      *pynvim*
-The Python client is the reference implementation for API clients. It is
-always up-to-date with the Nvim API, so its source code and test suite are
-authoritative references.
-    https://github.com/neovim/pynvim
-
-API client implementation guidelines ~
-
-  - Separate the transport layer from the rest of the library. See
-    |rpc-connecting| for details on how clients can connect to Nvim.
-  - Use a MessagePack library that implements at least version 5 of the
-    MessagePack spec, which supports the `bin` and `ext` types used by Nvim.
-  - Read API metadata in order to create client-side wrappers for all
-    msgpack-rpc methods.
-  - Use a single-threaded event loop library/pattern.
-  - Use a fiber/coroutine library for the language being used for implementing
-    a client. These greatly simplify concurrency and allow the library to
-    expose a blocking API on top of a non-blocking event loop without the
-    complexity that comes with preemptive multitasking.
-  - Don't assume anything about the order that responses to msgpack-rpc
-    requests will arrive.
-  - Clients should expect msgpack-rpc requests, which need to be handled
-    immediately because Nvim is blocked while waiting for the client response.
-  - Clients should expect to receive msgpack-rpc notifications, but these
-    don't need to be handled immediately because they won't block Nvim
-    (although they should probably be handled immediately anyway).
-
-Note: Most of the complexity could be handled by a msgpack-rpc library that
-supports server to client requests and notifications, but it's not clear if
-this is part of the msgpack-rpc spec. At least the Ruby msgpack-rpc library
-does not seem to support it:
-https://github.com/msgpack-rpc/msgpack-rpc-ruby/blob/master/lib/msgpack/rpc/transport/tcp.rb#L150-L158
-
-API metadata object ~
-
-API clients exist to hide msgpack-rpc details. The API metadata object
-contains information that makes this task easier (see also |rpc-types|):
-
-  - The "version" key contains the Nvim version, API level, and API
-    backwards-compatibility level.
-  - The "functions" key contains a list of metadata objects for individual
-    functions.
-  - Each function metadata object has |rpc-types| information about the return
-    value and parameters. These can be used for generating strongly-typed APIs
-    in static languages.
-  - Container types may be decorated with type/size constraints, e.g.
-    ArrayOf(Buffer) or ArrayOf(Integer, 2). This can be useful to generate
-    even more strongly-typed APIs.
-  - Functions that are considered to be methods that operate on instances of
-    Nvim special types (msgpack EXT) will have the `"method"` attribute set to
-    `true`. The receiver type is the type of the first argument. The method
-    names are prefixed with `nvim_` plus a shortened type name, e.g.
-    `nvim_buf_get_lines` represents the `get_lines` method of a Buffer instance.
-    - Global functions have `"method"` set to `false` and are prefixed with just
-    `nvim_`, e.g. `nvim_get_buffers`.
-
-So for an object-oriented language, an API client contains the classes
-representing Nvim special types, and the methods of each class could be
-defined by stripping the prefix for the type as defined in the `types` metadata
-(this will always be the first two "_"-separated parts of the function name).
-There could also be a singleton Vim class with methods where the `nvim_`
-prefix is stripped off.
-
-==============================================================================
-5. Types							    *rpc-types*
-
-The Nvim C API uses custom types for all functions. |api-types|
-At the RPC layer, types form two groups:
-
-  - Basic types that map natively to msgpack (and have a default
-    representation in most msgpack-supported programming languages)
-  - Special Nvim types that map to msgpack EXT with custom type codes.
-
-Basic types ~
-
-  Nil                               -> msgpack nil
-  Boolean                           -> msgpack boolean
-  Integer (signed 64-bit integer)   -> msgpack integer
-  Float (IEEE 754 double precision) -> msgpack float
-  String                            -> msgpack string
-  Array                             -> msgpack array
-  Dictionary                        -> msgpack map
-
-  Note: in function calls, empty Array is accepted for Dictionary parameter.
-
-Special types (msgpack EXT) ~
-
-  Buffer                            -> enum value kObjectTypeBuffer
-  Window                            -> enum value kObjectTypeWindow
-  Tabpage                           -> enum value kObjectTypeTabpage
-
-API functions expecting one of the special EXT types may be passed an integer
-instead, but not another EXT type. E.g. Buffer may be passed as an integer but
-not as a Window or Tabpage. The EXT object data is the object id encoded as
-a msgpack integer: For buffers this is the |bufnr()| and for windows the
-|window-ID|. For tabpages the id is an internal handle, not the tabpage
-number.
-
-To determine the type codes of the special EXT types, inspect the `types` key
-of the |api-metadata| at runtime. Example JSON representation: >
-
-  "types": {
-    "Buffer": {
-      "id": 0,
-      "prefix": "nvim_buf_"
-    },
-    "Window": {
-      "id": 1,
-      "prefix": "nvim_win_"
-    },
-    "Tabpage": {
-      "id": 2,
-      "prefix": "nvim_tabpage_"
-    }
-  }
-
-Even for statically compiled clients it is good practice to avoid hardcoding
-the type codes, because a client may be built against one Nvim version but
-connect to another with different type codes.
-
- vim:tw=78:ts=8:ft=help:norl:
+This document was merged into |api.txt| and |develop.txt|.
diff --git a/runtime/doc/starting.txt b/runtime/doc/starting.txt
index 1775fcddbd..7dbbb2d424 100644
--- a/runtime/doc/starting.txt
+++ b/runtime/doc/starting.txt
@@ -350,7 +350,7 @@ argument.
 
 							*--embed*
 --embed		Use stdin/stdout as a msgpack-RPC channel, so applications can
-		embed and control Nvim via the |rpc-api|.
+		embed and control Nvim via the RPC |API|.
 
 		Waits for the client ("embedder") to call |nvim_ui_attach()|
 		before sourcing startup files and reading buffers, so that UIs
diff --git a/runtime/doc/ui.txt b/runtime/doc/ui.txt
index 3b7debd99e..872049b356 100644
--- a/runtime/doc/ui.txt
+++ b/runtime/doc/ui.txt
@@ -4,12 +4,12 @@
 			    NVIM REFERENCE MANUAL
 
 
-Nvim UI protocol							  *ui*
+Nvim UI protocol					*UI* *ui*
 
                                       Type |gO| to see the table of contents.
 
 ==============================================================================
-UI Events							    *ui-events*
+UI Events						*ui-events*
 
 UIs can be implemented as external client processes communicating with Nvim
 over the RPC API. The default UI model is a terminal-like grid with a single,