diff options
Diffstat (limited to 'runtime/doc/msgpack_rpc.txt')
| -rw-r--r-- | runtime/doc/msgpack_rpc.txt | 323 |
1 files changed, 159 insertions, 164 deletions
diff --git a/runtime/doc/msgpack_rpc.txt b/runtime/doc/msgpack_rpc.txt index bafb9dfc2c..cfd9084cfc 100644 --- a/runtime/doc/msgpack_rpc.txt +++ b/runtime/doc/msgpack_rpc.txt @@ -1,97 +1,89 @@ -*msgpack_rpc.txt* For Nvim. {Nvim} +*msgpack_rpc.txt* {Nvim} NVIM REFERENCE MANUAL by Thiago de Arruda -The Msgpack-RPC Interface to Nvim *msgpack-rpc* +RPC API for Nvim *RPC* *rpc* *msgpack-rpc* -1. Introduction |msgpack-rpc-intro| -2. API mapping |msgpack-rpc-api| -3. Connecting |msgpack-rpc-connecting| -4. Clients |msgpack-rpc-clients| -5. Types |msgpack-rpc-types| -6. Wrapping methods |msgpack-rpc-wrap-methods| -7. Vimscript functions |msgpack-rpc-vim-functions| +1. Introduction |rpc-intro| +2. API mapping |rpc-api| +3. Connecting |rpc-connecting| +4. Clients |rpc-api-client| +5. Types |rpc-types| +6. Vimscript functions |rpc-vim-functions| ============================================================================== -1. Introduction *msgpack-rpc-intro* +1. Introduction *rpc-intro* -The primary way to control a running Nvim instance is through -MessagePack-RPC, a messaging protocol that uses the MessagePack serialization -format: https://github.com/msgpack/msgpack/blob/7498cf3/spec.md. -From now on, we refer to the protocol as msgpack-rpc. +The primary way to control Nvim programmatically is the RPC API, which speaks +MessagePack-RPC ("msgpack-rpc"), a messaging protocol that uses the +MessagePack serialization format: + https://github.com/msgpack/msgpack/blob/0b8f5ac/spec.md -At this point, only plugins use msgpack-rpc, but eventually even user -interaction will happen through it, since user interfaces will be separate -programs that control a headless Nvim instance. +All kinds of Nvim "clients" use the RPC API: user interfaces (GUIs), remote +plugins, scripts like "nvr" (https://github.com/mhinz/neovim-remote), and even +`nvim` itself can control other `nvim` instances. By connecting to the RPC API +programs can: -By connecting to the msgpack-rpc interface, programs can: + - Call any API function + - Listen for events + - Receive remote calls from Nvim -- Call any Nvim API function -- Listen for Nvim events -- Receive remote calls from Nvim - -Nvim's msgpack-rpc interface is like a more powerful version of Vim's -`clientserver` feature. +The RPC API is like a more powerful version of Vim's `clientserver` feature. ============================================================================== -2. API mapping *msgpack-rpc-api* + 2. API mapping *rpc-api* -The Nvim C API, see |nvim-api|, is automatically exposed to the msgpack-rpc -interface by the build system, which parses headers at src/nvim/api from the -project root. A dispatch function is generated, which matches msgpack-rpc method -names with non-static API functions, converting/validating arguments and return -values back to msgpack. +The Nvim C |API| is automatically exposed to the RPC API by the build system, +which parses headers at src/nvim/api/*. A dispatch function is generated which +matches RPC API method names with public API functions, converting/validating +arguments and return values back to msgpack. -Client libraries will normally provide wrappers that hide msgpack-rpc details -from programmers. The wrappers can be automatically generated by reading -bundled API metadata from a compiled Nvim instance. +Client libraries (|api-client|s) normally provide wrappers that hide +msgpack-rpc details from application developers. The wrappers can be +automatically generated by reading bundled API metadata from a compiled Nvim +instance. -There are two ways to obtain API metadata: +There are three ways to obtain API metadata: -1. By connecting to a running Nvim instance and calling `vim_get_api_info` - via msgpack-rpc. This is best for clients written in dynamically-typed - languages, which can define functions at runtime. + 1. Connect to a running Nvim instance and call `vim_get_api_info` via + msgpack-rpc. This is best for clients written in dynamic languages which + can define functions at runtime. -2. By starting Nvim with the `--api-info` command-line option, which makes Nvim - dump a blob of msgpack metadata to standard output and exit. This is best - for clients written in statically-typed languages, which require a separate - compilation step. + 2. Start Nvim with the |--api-info| option. Useful for clients written in + statically-compiled languages. -Here's a simple way to get human-readable description of the API (requires -Python and the `pyyaml`/`msgpack-python` pip packages): -> - nvim --api-info | python -c 'import msgpack, sys, yaml; print yaml.dump(msgpack.unpackb(sys.stdin.read()))' > api.yaml + 3. Use the |api_info()| vimscript function. + +To get a human-readable list of API functions: > + :new|put =map(api_info().functions, 'v:val.name') +< +To get a formatted dump of the API using python (requires the `pyyaml` and +`msgpack-python` packages): > + nvim --api-info | python -c 'import msgpack, sys, yaml; print yaml.dump(msgpack.unpackb(sys.stdin.read()))' < ============================================================================== -3. Connecting *msgpack-rpc-connecting* +3. Connecting *rpc-connecting* -There are four ways to open msgpack-rpc streams to Nvim: +There are several ways to open a msgpack-rpc stream to an Nvim server: -1. Through Nvim's stdin/stdout when it's started with the `--embed` option. - This is how other programs can embed Nvim. + 1. Through stdin/stdout when `nvim` is started with `--embed`. This is how + applications can embed Nvim. -2. Through the stdin/stdout of a program spawned by the |rpcstart()| function. + 2. Through stdin/stdout of some other process spawned by |rpcstart()|. - *$NVIM_LISTEN_ADDRESS* -3. Through the socket automatically created with each instance. To get the - socket location for a running Nvim instance (which is random by default), - see the |$NVIM_LISTEN_ADDRESS| environment variable: -> - :echo $NVIM_LISTEN_ADDRESS -< - See also |v:servername|. + 3. Through the socket automatically created with each instance. The socket + location is stored in |v:servername|. -4. Through a TCP/IP socket. To make Nvim listen on a TCP/IP socket, set the - |$NVIM_LISTEN_ADDRESS| environment variable in a shell before starting Nvim: -> + 4. Through a TCP/IP socket. To make Nvim listen on a TCP/IP socket, set the + |$NVIM_LISTEN_ADDRESS| environment variable before starting Nvim: > NVIM_LISTEN_ADDRESS=127.0.0.1:6666 nvim < -Connecting to the socket is the easiest way a programmer can test the API, which -can be done through any msgpack-rpc client library or fully-featured Nvim client -(which we'll see in the next section). Here's a Ruby script that prints 'hello -world!' in the current Nvim instance: +Connecting to the socket is the easiest way a programmer can test the API, +which can be done through any msgpack-rpc client library or full-featured +|api-client|. Here's a Ruby script that prints 'hello world!' in the current +Nvim instance: > #!/usr/bin/env ruby # Requires msgpack-rpc: gem install msgpack-rpc @@ -118,72 +110,103 @@ functions can be called interactively: >>> nvim = attach('socket', path='[address]') >>> nvim.command('echo "hello world!"') < -One can also spawn and connect to an embedded Nvim instance via |rpcstart()| +You can also embed an Nvim instance via |rpcstart()| > let vim = rpcstart('nvim', ['--embed']) echo rpcrequest(vim, 'vim_eval', '"Hello " . "world!"') call rpcstop(vim) < ============================================================================== -4. Implementing new clients *msgpack-rpc-clients* - -Nvim is still in alpha, so there's no in-depth documentation explaining how to -properly implement a client library yet. The Python client (the pip package -"neovim") will always be up-to-date with the latest API changes, so its source -code is the best documentation currently available. There are some guidelines -however: - -- Separate the transport layer from the rest of the library. See - |msgpack-rpc-connecting| for details on how clients can connect to Nvim. -- Use a MessagePack library that implements at least version 5 of the - MessagePack spec, which supports the `bin` and `ext` types used by Nvim. -- Read API metadata in order to create client-side wrappers for all - msgpack-rpc methods. -- Use a single-threaded event loop library/pattern. -- Use a fiber/coroutine library for the language being used for implementing a - client. These greatly simplify concurrency and allow the library to expose a - blocking API on top of a non-blocking event loop without the complexity that - comes with preemptive multitasking. -- Don't assume anything about the order that responses to msgpack-rpc requests - will arrive. -- Clients should expect msgpack-rpc requests, which need to be handled - immediately because Nvim is blocked while waiting for the client response. -- Clients should expect to receive msgpack-rpc notifications, but these don't - need to be handled immediately because they won't block Nvim (although they - should probably be handled immediately anyway). - -Most of the complexity could be handled by a msgpack-rpc library that supports -server to client requests and notifications, but it's not clear if this is part -of the msgpack-rpc spec. At least the Ruby msgpack-rpc library does not seem -to support it: - +4. Implementing API clients *rpc-api-client* *api-client* + +All external UIs and remote plugins (as opposed to regular Vim plugins) are +"clients" in general; but we call something an "API client" if its purpose is +to abstract or wrap the RPC API for the convenience of other applications +(just like a REST client or SDK such as boto3 for AWS: you can speak AWS REST +using an HTTP client like curl, but boto3 wraps that in a convenient python +interface). For example, the lua-client is an API client: + https://github.com/neovim/lua-client + +The Python client (pip package "neovim") is the reference implementation of an +API client. It is always up-to-date with the Nvim API, so its source code and +test suite are an authoritative reference. + https://github.com/neovim/python-client + +API client implementation guidelines ~ + + - Separate the transport layer from the rest of the library. See + |rpc-connecting| for details on how clients can connect to Nvim. + - Use a MessagePack library that implements at least version 5 of the + MessagePack spec, which supports the `bin` and `ext` types used by Nvim. + - Read API metadata in order to create client-side wrappers for all + msgpack-rpc methods. + - Use a single-threaded event loop library/pattern. + - Use a fiber/coroutine library for the language being used for implementing + a client. These greatly simplify concurrency and allow the library to + expose a blocking API on top of a non-blocking event loop without the + complexity that comes with preemptive multitasking. + - Don't assume anything about the order that responses to msgpack-rpc + requests will arrive. + - Clients should expect msgpack-rpc requests, which need to be handled + immediately because Nvim is blocked while waiting for the client response. + - Clients should expect to receive msgpack-rpc notifications, but these + don't need to be handled immediately because they won't block Nvim + (although they should probably be handled immediately anyway). + +Note: Most of the complexity could be handled by a msgpack-rpc library that +supports server to client requests and notifications, but it's not clear if +this is part of the msgpack-rpc spec. At least the Ruby msgpack-rpc library +does not seem to support it: https://github.com/msgpack-rpc/msgpack-rpc-ruby/blob/master/lib/msgpack/rpc/transport/tcp.rb#L150-L158 +API metadata object ~ + +API clients exist to hide msgpack-rpc details. The API metadata object +contains information that makes this task easier (see also |rpc-types|): + + - The "functions" key contains a list of metadata objects for individual + functions. + - Each function metadata object has |rpc-types| information about the return + value and parameters. These can be used for generating strongly-typed APIs + in static languages. + - Container types may be decorated with type/size constraints, e.g. + ArrayOf(Buffer) or ArrayOf(Integer, 2). This can be useful to generate + even more strongly-typed APIs. + - Methods that operate on instances of Nvim special types (msgpack EXT) are + prefixed with the type name in lower case, e.g. `buffer_get_line` + represents the `get_line` method of a Buffer instance. + - Global methods are prefixed with `vim`, e.g. `vim_get_buffers`. + +So for an object-oriented language, an API client contains the classes +representing Nvim special types, and the methods of each class could be +defined by inspecting the method name prefix. There could also be a singleton +Vim class with methods mapped to functions prefixed with `vim_`. + ============================================================================== -5. Types *msgpack-rpc-types* +5. Types *rpc-types* -Nvim's C API uses custom types for all functions, se |nvim-api-types|. -For the purpose of mapping to msgpack, he types can be split into two groups: +The Nvim C API uses custom types for all functions. |api-types| +For the purpose of mapping to msgpack, the types can be split into two groups: -- Basic types that map natively to msgpack (and probably have a default - representation in msgpack-supported programming languages) -- Special Nvim types that map to msgpack EXT with custom type codes. + - Basic types that map natively to msgpack (and probably have a default + representation in msgpack-supported programming languages) + - Special Nvim types that map to msgpack EXT with custom type codes. -Basic type mapping: +Basic types ~ -Nil -> msgpack nil -Boolean -> msgpack boolean -Integer (signed 64-bit integer) -> msgpack integer -Float (IEEE 754 double precision) -> msgpack float -String -> msgpack string -Array -> msgpack array -Dictionary -> msgpack map + Nil -> msgpack nil + Boolean -> msgpack boolean + Integer (signed 64-bit integer) -> msgpack integer + Float (IEEE 754 double precision) -> msgpack float + String -> msgpack string + Array -> msgpack array + Dictionary -> msgpack map -Special Nvim types that use msgpack EXT: +Special types (msgpack EXT) ~ -Buffer -> enum value kObjectTypeBuffer -Window -> enum value kObjectTypeWindow -Tabpage -> enum value kObjectTypeTabpage + Buffer -> enum value kObjectTypeBuffer + Window -> enum value kObjectTypeWindow + Tabpage -> enum value kObjectTypeTabpage An API method expecting one of these types may be passed an integer instead, although they are not interchangeable. For example, a Buffer may be passed as @@ -191,7 +214,7 @@ an integer, but not a Window or Tabpage. The most reliable way of determining the type codes for the special Nvim types is to inspect the `types` key of metadata dictionary returned by the -`vim_get_api_info` method at runtime. Here's an example JSON representation of +`vim_get_api_info` method at runtime. Here's a sample JSON representation of the `types` object: > "types": { @@ -206,55 +229,27 @@ the `types` object: } } < -Even for statically compiled clients, it's a good practice to avoid hardcoding -the type codes, because a client may be built against one Nvim version but connect -to another with different type codes. +Even for statically compiled clients it is good practice to avoid hardcoding +the type codes, because a client may be built against one Nvim version but +connect to another with different type codes. ============================================================================== -6. Wrapping methods *msgpack-rpc-wrap-methods* - -As mentioned before, clients should provide an API that hides msgpack-rpc -details from programmers, and the API metadata object contains information -that makes this task easier: - -- The "functions" key contains a list of metadata objects for individual - functions. -- Each function metadata object has type information about the return value - and parameters. These can be used for generating strongly-typed APIs in - static languages. -- Container types may be decorated with type/size constraints, e.g. - ArrayOf(Buffer) or ArrayOf(Integer, 2). This can be useful to generate even - more strongly-typed APIs. -- Methods that operate instances of Nvim's types are prefixed with the type - name in lower case, e.g. `buffer_get_line` represents the `get_line` method - of a Buffer instance. -- Global methods are prefixed with `vim`, e.g. `vim_get_buffers`. - -So, for an object-oriented language, a client library would have the classes -that represent Nvim's types, and the methods of each class could be defined -by inspecting the method name prefix. There could also be a singleton Vim -class with methods mapped to functions prefixed with `vim_` - -============================================================================== -7. Vimscript functions *msgpack-rpc-vim-functions* - -Four msgpack-rpc functions are available in Vimscript: - -1. |rpcstart()|: Similarly to |jobstart()|, this will spawn a co-process with - its standard handles connected to Nvim. The difference is that it's not - possible to process raw data to or from the process's stdin, stdout, or - stderr. This is because the job's stdin and stdout are used as a single - msgpack channel that is processed directly by Nvim. - -2. |rpcstop()|: Same as |jobstop()|, but operates on handles returned by - |rpcstart()|. - -3. |rpcrequest()|: Sends a msgpack-rpc request to the process. - -4. |rpcnotify()|: Sends a msgpack-rpc notification to the process. - -The last two functions may also be used with channels created from -connections to |$NVIM_LISTEN_ADDRESS|. +6. Vimscript functions *rpc-vim-functions* + +RPC functions are available in Vimscript: + + 1. |rpcstart()|: Similarly to |jobstart()|, this will spawn a co-process + with its standard handles connected to Nvim. The difference is that it's + not possible to process raw data to or from the process's stdin, stdout, + or stderr. This is because the job's stdin and stdout are used as + a single msgpack channel that is processed directly by Nvim. + 2. |rpcstop()|: Same as |jobstop()|, but operates on handles returned by + |rpcstart()|. + 3. |rpcrequest()|: Sends a msgpack-rpc request to the process. + 4. |rpcnotify()|: Sends a msgpack-rpc notification to the process. + +|rpcrequest()| and |rpcnotify()| can also be used with channels connected to +a nvim server. |v:servername| ============================================================================== vim:tw=78:ts=8:noet:ft=help:norl: |