1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
|
*msgpack_rpc.txt* {Nvim}
NVIM REFERENCE MANUAL by Thiago de Arruda
RPC API for Nvim *RPC* *rpc* *msgpack-rpc*
1. Introduction |rpc-intro|
2. API mapping |rpc-api|
3. Connecting |rpc-connecting|
4. Clients |rpc-client|
5. Types |rpc-types|
6. Vimscript functions |rpc-vim-functions|
==============================================================================
1. Introduction *rpc-intro*
The primary way to control Nvim programmatically is the RPC API, which speaks
MessagePack-RPC ("msgpack-rpc"), a messaging protocol that uses the
MessagePack serialization format:
https://github.com/msgpack/msgpack/blob/0b8f5ac/spec.md
All kinds of Nvim "clients" use the RPC API: user interfaces (GUIs), remote
plugins, scripts like "nvr" (https://github.com/mhinz/neovim-remote), and even
`nvim` itself can control other `nvim` instances. By connecting to the RPC API
programs can:
- Call any API function
- Listen for events
- Receive remote calls from Nvim
The RPC API is like a more powerful version of Vim's `clientserver` feature.
==============================================================================
2. API mapping *rpc-api*
The Nvim C |API| is automatically exposed to the RPC API by the build system,
which parses headers at src/nvim/api/*. A dispatch function is generated which
matches RPC API method names with public API functions, converting/validating
arguments and return values back to msgpack.
Client libraries (|api-client|s) normally provide wrappers that hide
msgpack-rpc details from application developers. The wrappers can be
automatically generated by reading bundled API metadata from a compiled Nvim
instance.
There are two ways to obtain API metadata:
1. Connect to a running Nvim instance and call `vim_get_api_info` via
msgpack-rpc. This is best for clients written in dynamic languages which
can define functions at runtime.
2. Start Nvim with the `--api-info` command-line option, which dumps a blob
of msgpack metadata to standard output. This is useful for clients
written in statically-compiled languages.
To get a human-readable list of API functions: >
:new|put =map(msgpackparse(systemlist('nvim --api-info'))[0].functions, 'v:val.name._VAL[0]')
<
To get a formatted dump of the API using python (requires the `pyyaml` and
`msgpack-python` packages): >
nvim --api-info | python -c 'import msgpack, sys, yaml; print yaml.dump(msgpack.unpackb(sys.stdin.read()))'
<
==============================================================================
3. Connecting *rpc-connecting*
There are several ways to open a msgpack-rpc stream to an Nvim server:
1. Through stdin/stdout when `nvim` is started with `--embed`. This is how
applications can embed Nvim.
2. Through stdin/stdout of some other process spawned by |rpcstart()|.
3. Through the socket automatically created with each instance. The socket
location is stored in |v:servername|.
4. Through a TCP/IP socket. To make Nvim listen on a TCP/IP socket, set the
|$NVIM_LISTEN_ADDRESS| environment variable before starting Nvim: >
NVIM_LISTEN_ADDRESS=127.0.0.1:6666 nvim
<
Connecting to the socket is the easiest way a programmer can test the API,
which can be done through any msgpack-rpc client library or full-featured
|api-client|. Here's a Ruby script that prints 'hello world!' in the current
Nvim instance:
>
#!/usr/bin/env ruby
# Requires msgpack-rpc: gem install msgpack-rpc
#
# To run this script, execute it from a running Nvim instance (notice the
# trailing '&' which is required since Nvim won't process events while
# running a blocking command):
#
# :!./hello.rb &
#
# Or from another shell by setting NVIM_LISTEN_ADDRESS:
# $ NVIM_LISTEN_ADDRESS=[address] ./hello.rb
require 'msgpack/rpc'
require 'msgpack/rpc/transport/unix'
nvim = MessagePack::RPC::Client.new(MessagePack::RPC::UNIXTransport.new, ENV['NVIM_LISTEN_ADDRESS'])
result = nvim.call(:vim_command, 'echo "hello world!"')
<
A better way is to use the Python REPL with the `neovim` package, where API
functions can be called interactively:
>
>>> from neovim import attach
>>> nvim = attach('socket', path='[address]')
>>> nvim.command('echo "hello world!"')
<
You can also embed an Nvim instance via |rpcstart()|
>
let vim = rpcstart('nvim', ['--embed'])
echo rpcrequest(vim, 'vim_eval', '"Hello " . "world!"')
call rpcstop(vim)
<
==============================================================================
4. Implementing API clients *rpc-api-client* *api-client*
All external UIs and remote plugins (as opposed to regular Vim plugins) are
"clients" in general; but we call something an "API client" if its purpose is
to abstract or wrap the RPC API for the convenience of other applications
(just like a REST client or SDK such as boto3 for AWS: you can speak AWS REST
using an HTTP client like curl, but boto3 wraps that in a convenient python
interface). For example, the lua-client is an API client:
https://github.com/neovim/lua-client
The Python client (pip package "neovim") is the reference implementation of an
API client. It is always up-to-date with the Nvim API, so its source code and
test suite are an authoritative reference.
https://github.com/neovim/python-client
API client implementation guidelines ~
- Separate the transport layer from the rest of the library. See
|rpc-connecting| for details on how clients can connect to Nvim.
- Use a MessagePack library that implements at least version 5 of the
MessagePack spec, which supports the `bin` and `ext` types used by Nvim.
- Read API metadata in order to create client-side wrappers for all
msgpack-rpc methods.
- Use a single-threaded event loop library/pattern.
- Use a fiber/coroutine library for the language being used for implementing
a client. These greatly simplify concurrency and allow the library to
expose a blocking API on top of a non-blocking event loop without the
complexity that comes with preemptive multitasking.
- Don't assume anything about the order that responses to msgpack-rpc
requests will arrive.
- Clients should expect msgpack-rpc requests, which need to be handled
immediately because Nvim is blocked while waiting for the client response.
- Clients should expect to receive msgpack-rpc notifications, but these
don't need to be handled immediately because they won't block Nvim
(although they should probably be handled immediately anyway).
Note: Most of the complexity could be handled by a msgpack-rpc library that
supports server to client requests and notifications, but it's not clear if
this is part of the msgpack-rpc spec. At least the Ruby msgpack-rpc library
does not seem to support it:
https://github.com/msgpack-rpc/msgpack-rpc-ruby/blob/master/lib/msgpack/rpc/transport/tcp.rb#L150-L158
API metadata object ~
API clients exist to hide msgpack-rpc details. The API metadata object
contains information that makes this task easier (see also |rpc-types|):
- The "functions" key contains a list of metadata objects for individual
functions.
- Each function metadata object has |rpc-types| information about the return
value and parameters. These can be used for generating strongly-typed APIs
in static languages.
- Container types may be decorated with type/size constraints, e.g.
ArrayOf(Buffer) or ArrayOf(Integer, 2). This can be useful to generate
even more strongly-typed APIs.
- Methods that operate on instances of Nvim special types (msgpack EXT) are
prefixed with the type name in lower case, e.g. `buffer_get_line`
represents the `get_line` method of a Buffer instance.
- Global methods are prefixed with `vim`, e.g. `vim_get_buffers`.
So for an object-oriented language, an API client contains the classes
representing Nvim special types, and the methods of each class could be
defined by inspecting the method name prefix. There could also be a singleton
Vim class with methods mapped to functions prefixed with `vim_`.
==============================================================================
5. Types *rpc-types*
The Nvim C API uses custom types for all functions. |api-types|
For the purpose of mapping to msgpack, the types can be split into two groups:
- Basic types that map natively to msgpack (and probably have a default
representation in msgpack-supported programming languages)
- Special Nvim types that map to msgpack EXT with custom type codes.
Basic types ~
Nil -> msgpack nil
Boolean -> msgpack boolean
Integer (signed 64-bit integer) -> msgpack integer
Float (IEEE 754 double precision) -> msgpack float
String -> msgpack string
Array -> msgpack array
Dictionary -> msgpack map
Special types (msgpack EXT) ~
Buffer -> enum value kObjectTypeBuffer
Window -> enum value kObjectTypeWindow
Tabpage -> enum value kObjectTypeTabpage
An API method expecting one of these types may be passed an integer instead,
although they are not interchangeable. For example, a Buffer may be passed as
an integer, but not a Window or Tabpage.
The most reliable way of determining the type codes for the special Nvim types
is to inspect the `types` key of metadata dictionary returned by the
`vim_get_api_info` method at runtime. Here's a sample JSON representation of
the `types` object:
>
"types": {
"Buffer": {
"id": 0
},
"Window": {
"id": 1
},
"Tabpage": {
"id": 2
}
}
<
Even for statically compiled clients it is good practice to avoid hardcoding
the type codes, because a client may be built against one Nvim version but
connect to another with different type codes.
==============================================================================
6. Vimscript functions *rpc-vim-functions*
RPC functions are available in Vimscript:
1. |rpcstart()|: Similarly to |jobstart()|, this will spawn a co-process
with its standard handles connected to Nvim. The difference is that it's
not possible to process raw data to or from the process's stdin, stdout,
or stderr. This is because the job's stdin and stdout are used as
a single msgpack channel that is processed directly by Nvim.
2. |rpcstop()|: Same as |jobstop()|, but operates on handles returned by
|rpcstart()|.
3. |rpcrequest()|: Sends a msgpack-rpc request to the process.
4. |rpcnotify()|: Sends a msgpack-rpc notification to the process.
|rpcrequest()| and |rpcnotify()| can also be used with channels connected to
a nvim server. |v:servername|
==============================================================================
vim:tw=78:ts=8:noet:ft=help:norl:
|
