diff options
60 files changed, 3582 insertions, 300 deletions
diff --git a/BSDmakefile b/BSDmakefile index 93b7dc7f3d..f81223bafa 100644 --- a/BSDmakefile +++ b/BSDmakefile @@ -1,4 +1,4 @@ .DONE: - @echo "Please use GNU Make (gmake) to build neovim" + @echo "Use GNU Make (gmake) to build neovim" .DEFAULT: - @echo "Please use GNU Make (gmake) to build neovim" + @echo "Use GNU Make (gmake) to build neovim" diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4d93c979ed..2a565574fa 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -29,10 +29,10 @@ Reporting problems Developer guidelines -------------------- -- Nvim developers should read `:help dev`. -- External UI developers should read `:help dev-ui`. -- API client developers should read `:help dev-api-client`. -- Nvim developers are _strongly encouraged_ to install `ninja` for faster builds. +- Read `:help dev` if you are working on Nvim core. +- Read `:help dev-ui` if you are developing a UI. +- Read `:help dev-api-client` if you are developing an API client. +- Install `ninja` for faster builds of Nvim. ``` sudo apt-get install ninja-build make distclean diff --git a/man/nvim.1 b/man/nvim.1 index ca0f41d489..3f57f397d2 100644 --- a/man/nvim.1 +++ b/man/nvim.1 @@ -100,7 +100,7 @@ Useful for scripting because it does NOT start a UI, unlike .Ic :help silent-mode .It Fl d Diff mode. -Show the difference between two to four files, similar to +Show the difference between two to eight files, similar to .Xr sdiff 1 . .Ic ":help diff" .It Fl R diff --git a/runtime/autoload/rubycomplete.vim b/runtime/autoload/rubycomplete.vim index e8a1879668..3677b25aeb 100644 --- a/runtime/autoload/rubycomplete.vim +++ b/runtime/autoload/rubycomplete.vim @@ -3,7 +3,7 @@ " Maintainer: Mark Guzman <segfault@hasno.info> " URL: https://github.com/vim-ruby/vim-ruby " Release Coordinator: Doug Kearns <dougkearns@gmail.com> -" Last Change: 2019 Feb 25 +" Last Change: 2020 Apr 12 " ---------------------------------------------------------------------------- " " Ruby IRB/Complete author: Keiju ISHITSUKA(keiju@ishitsuka.com) @@ -501,13 +501,8 @@ class VimRubyCompletion return if rails_base == nil $:.push rails_base unless $:.index( rails_base ) - rails_config = rails_base + "config/" - rails_lib = rails_base + "lib/" - $:.push rails_config unless $:.index( rails_config ) - $:.push rails_lib unless $:.index( rails_lib ) - - bootfile = rails_config + "boot.rb" - envfile = rails_config + "environment.rb" + bootfile = rails_base + "config/boot.rb" + envfile = rails_base + "config/environment.rb" if File.exists?( bootfile ) && File.exists?( envfile ) begin require bootfile diff --git a/runtime/doc/arabic.txt b/runtime/doc/arabic.txt index df91b8d065..5d3bf7a761 100644 --- a/runtime/doc/arabic.txt +++ b/runtime/doc/arabic.txt @@ -171,6 +171,13 @@ o Enable Arabic settings [short-cut] and its support is preferred due to its level of offerings. 'arabic' when 'termbidi' is enabled only sets the keymap. + For vertical window isolation while setting 'termbidi' an LTR + vertical separator like "l" or "𝖨" may be used. It may also be + hidden by changing its color to the foreground color: > + :set fillchars=vert:l + :hi VertSplit ctermbg=White +< Note that this is a workaround, not a proper solution. + If, on the other hand, you'd like to be verbose and explicit and are opting not to use the 'arabic' short-cut command, here's what is needed (i.e. if you use ':set arabic' you can skip this section) - diff --git a/runtime/doc/change.txt b/runtime/doc/change.txt index aed3acab67..2b799e3e27 100644 --- a/runtime/doc/change.txt +++ b/runtime/doc/change.txt @@ -747,12 +747,14 @@ For compatibility with Vi these two exceptions are allowed: "\/{string}/" and "\?{string}?" do the same as "//{string}/r". "\&{string}&" does the same as "//{string}/". *pattern-delimiter* *E146* -Instead of the '/' which surrounds the pattern and replacement string, you -can use any other single-byte character, but not an alphanumeric character, -'\', '"' or '|'. This is useful if you want to include a '/' in the search -pattern or replacement string. Example: > +Instead of the '/' which surrounds the pattern and replacement string, you can +use another single-byte character. This is useful if you want to include a +'/' in the search pattern or replacement string. Example: > :s+/+//+ +You can use most characters, but not an alphanumeric character, '\', '"' or +'|'. + For the definition of a pattern, see |pattern|. In Visual block mode, use |/\%V| in the pattern to have the substitute work in the block only. Otherwise it works on whole lines anyway. diff --git a/runtime/doc/deprecated.txt b/runtime/doc/deprecated.txt index 3b5287ee44..861aed4884 100644 --- a/runtime/doc/deprecated.txt +++ b/runtime/doc/deprecated.txt @@ -14,6 +14,7 @@ updated. API ~ *nvim_buf_clear_highlight()* Use |nvim_buf_clear_namespace()| instead. +*nvim_buf_set_virtual_text()* Use |nvim_buf_set_extmark()| instead. *nvim_command_output()* Use |nvim_exec()| instead. *nvim_execute_lua()* Use |nvim_exec_lua()| instead. @@ -54,6 +55,8 @@ Functions ~ without stopping the job. Use chanclose(id) to close any socket. +Lua ~ +*vim.register_keystroke_callback()* Use |vim.on_key()| instead. Modifiers ~ *cpo-<* diff --git a/runtime/doc/develop.txt b/runtime/doc/develop.txt index 60a3f870a9..14f35acce3 100644 --- a/runtime/doc/develop.txt +++ b/runtime/doc/develop.txt @@ -127,14 +127,20 @@ Sometimes a GUI or other application may want to force a provider to DOCUMENTATION *dev-doc* -- Do not prefix help tags with "nvim-". Use |vim_diff.txt| to document - differences from Vim; no other distinction is necessary. -- If a Vim feature is removed, delete its help section and move its tag to - |vim_diff.txt|. -- Move deprecated features to |deprecated.txt|. +- "Just say it". Avoid mushy, colloquial phrasing in all documentation + (docstrings, user manual, website materials, newsletters, …). Don't mince + words. Personality and flavor, used sparingly, are welcome--but in general, + optimize for the reader's time and energy: be "precise yet concise". + - Prefer the active voice: "Foo does X", not "X is done by Foo". +- Vim differences: + - Do not prefix help tags with "nvim-". Use |vim_diff.txt| to catalog + differences from Vim; no other distinction is necessary. + - If a Vim feature is removed, delete its help section and move its tag to + |vim_diff.txt|. +- Mention deprecated features in |deprecated.txt| and delete their old doc. - Use consistent language. - - "terminal" in a help tag always means "the embedded terminal emulator", not - "the user host terminal". + - "terminal" in a help tag always means "the embedded terminal emulator", + not "the user host terminal". - Use "tui-" to prefix help tags related to the host terminal, and "TUI" in prose if possible. - Docstrings: do not start parameter descriptions with "The" or "A" unless it @@ -222,13 +228,13 @@ LUA *dev-lua* - Keep the core Lua modules |lua-stdlib| simple. Avoid elaborate OOP or pseudo-OOP designs. Plugin authors just want functions to call, they don't - want to learn a big, fancy inheritance hierarchy. So we should avoid complex - objects: tables are usually better. + want to learn a big, fancy inheritance hierarchy. Thus avoid specialized + objects; tables or values are usually better. API *dev-api* -Use this template to name new API functions: +Use this template to name new RPC |API| functions: nvim_{thing}_{action}_{arbitrary-qualifiers} If the function acts on an object then {thing} is the name of that object @@ -356,4 +362,19 @@ External UIs are expected to implement these common features: published in this event. See also "mouse_on", "mouse_off". +NAMING *dev-naming* + +Naming is important. Consistent naming in the API and UI helps both users and +developers discover and intuitively understand related concepts ("families"), +and reduces cognitive burden. Discoverability encourages code re-use and +likewise avoids redundant, overlapping mechanisms, which reduces code +surface-area, and thereby minimizes bugs... + +Naming conventions ~ + +Use the "on_" prefix to name event handlers and also the interface for +"registering" such handlers (on_key). The dual nature is acceptable to avoid +a confused collection of naming conventions for these related concepts. + + vim:tw=78:ts=8:noet:ft=help:norl: diff --git a/runtime/doc/eval.txt b/runtime/doc/eval.txt index e6405145cd..5d889983e3 100644 --- a/runtime/doc/eval.txt +++ b/runtime/doc/eval.txt @@ -1856,7 +1856,7 @@ v:null Special value used to put "null" in JSON and NIL in msgpack. v:numbermax Maximum value of a number. *v:numbermin* *numbermin-variable* -v:numbermin Minimum value of a number (negative) +v:numbermin Minimum value of a number (negative). *v:numbersize* *numbersize-variable* v:numbersize Number of bits in a Number. This is normally 64, but on some @@ -2149,7 +2149,7 @@ chanclose({id}[, {stream}]) Number Closes a channel or one of its streams chansend({id}, {data}) Number Writes {data} to channel char2nr({expr}[, {utf8}]) Number ASCII/UTF8 value of first char in {expr} charidx({string}, {idx} [, {countcc}]) - Number char index of byte {idx} in {string} + Number char index of byte {idx} in {string} cindent({lnum}) Number C indent for line {lnum} clearmatches([{win}]) none clear all matches col({expr}) Number column nr of cursor or mark @@ -4196,11 +4196,15 @@ foldclosed({lnum}) *foldclosed()* The result is a Number. If the line {lnum} is in a closed fold, the result is the number of the first line in that fold. If the line {lnum} is not in a closed fold, -1 is returned. + {lnum} is used like with |getline()|. Thus "." is the current + line, "'m" mark m, etc. foldclosedend({lnum}) *foldclosedend()* The result is a Number. If the line {lnum} is in a closed fold, the result is the number of the last line in that fold. If the line {lnum} is not in a closed fold, -1 is returned. + {lnum} is used like with |getline()|. Thus "." is the current + line, "'m" mark m, etc. foldlevel({lnum}) *foldlevel()* The result is a Number, which is the foldlevel of line {lnum} @@ -4211,6 +4215,8 @@ foldlevel({lnum}) *foldlevel()* returned for lines where folds are still to be updated and the foldlevel is unknown. As a special case the level of the previous line is usually available. + {lnum} is used like with |getline()|. Thus "." is the current + line, "'m" mark m, etc. *foldtext()* foldtext() Returns a String, to be displayed for a closed fold. This is @@ -4862,11 +4868,11 @@ getmarklist([{expr}]) *getmarklist()* see |bufname()|. Each item in the returned List is a |Dict| with the following: - name - name of the mark prefixed by "'" - pos - a |List| with the position of the mark: + mark name of the mark prefixed by "'" + pos a |List| with the position of the mark: [bufnum, lnum, col, off] - Refer to |getpos()| for more information. - file - file name + Refer to |getpos()| for more information. + file file name Refer to |getpos()| for getting information about a specific mark. @@ -4877,6 +4883,8 @@ getmatches([{win}]) *getmatches()* |getmatches()| is useful in combination with |setmatches()|, as |setmatches()| can restore a list of matches saved by |getmatches()|. + If {win} is specified, use the window with this number or + window ID instead of the current window. Example: > :echo getmatches() < [{'group': 'MyGroup1', 'pattern': 'TODO', @@ -4945,8 +4953,10 @@ getqflist([{what}]) *getqflist()* valid |TRUE|: recognized error message When there is no error list or it's empty, an empty list is - returned. Quickfix list entries with non-existing buffer - number are returned with "bufnr" set to zero. + returned. Quickfix list entries with a non-existing buffer + number are returned with "bufnr" set to zero (Note: some + functions accept buffer number zero for the alternate buffer, + you may need to explicitly check for zero). Useful application: Find pattern matches in multiple files and do something with them: > @@ -5050,12 +5060,12 @@ getregtype([{regname}]) *getregtype()* <CTRL-V> is one character with value 0x16. If {regname} is not specified, |v:register| is used. -gettabinfo([{arg}]) *gettabinfo()* - If {arg} is not specified, then information about all the tab - pages is returned as a |List|. Each List item is a |Dictionary|. - Otherwise, {arg} specifies the tab page number and information - about that one is returned. If the tab page does not exist an - empty List is returned. +gettabinfo([{tabnr}]) *gettabinfo()* + If {tabnr} is not specified, then information about all the + tab pages is returned as a |List|. Each List item is a + |Dictionary|. Otherwise, {tabnr} specifies the tab page + number and information about that one is returned. If the tab + page does not exist an empty List is returned. Each List item is a |Dictionary| with the following entries: tabnr tab page number. @@ -5099,11 +5109,11 @@ gettabwinvar({tabnr}, {winnr}, {varname} [, {def}]) *gettabwinvar()* To obtain all window-local variables use: > gettabwinvar({tabnr}, {winnr}, '&') -gettagstack([{nr}]) *gettagstack()* - The result is a Dict, which is the tag stack of window {nr}. - {nr} can be the window number or the |window-ID|. - When {nr} is not specified, the current window is used. - When window {nr} doesn't exist, an empty Dict is returned. +gettagstack([{winnr}]) *gettagstack()* + The result is a Dict, which is the tag stack of window {winnr}. + {winnr} can be the window number or the |window-ID|. + When {winnr} is not specified, the current window is used. + When window {winnr} doesn't exist, an empty Dict is returned. The returned dictionary contains the following entries: curidx Current index in the stack. When at diff --git a/runtime/doc/filetype.txt b/runtime/doc/filetype.txt index 36ed6bbac1..894d1627ab 100644 --- a/runtime/doc/filetype.txt +++ b/runtime/doc/filetype.txt @@ -580,7 +580,7 @@ To disable bold highlighting: > MARKDOWN *ft-markdown-plugin* To enable folding use this: > - let g:markdown_folding = 1 + let g:markdown_folding = 1 < PDF *ft-pdf-plugin* diff --git a/runtime/doc/ft_ps1.txt b/runtime/doc/ft_ps1.txt index df1480b929..3eb89a4c24 100644 --- a/runtime/doc/ft_ps1.txt +++ b/runtime/doc/ft_ps1.txt @@ -1,4 +1,4 @@ -*ps1.txt* A Windows PowerShell syntax plugin for Vim +*ft_ps1.txt* A Windows PowerShell syntax plugin for Vim Author: Peter Provost <https://www.github.com/PProvost> License: Apache 2.0 diff --git a/runtime/doc/ft_raku.txt b/runtime/doc/ft_raku.txt index 26ada8a140..00b140ee9c 100644 --- a/runtime/doc/ft_raku.txt +++ b/runtime/doc/ft_raku.txt @@ -1,4 +1,4 @@ -*vim-raku.txt* The Raku programming language filetype +*ft_raku.txt* The Raku programming language filetype *vim-raku* @@ -45,7 +45,7 @@ Numbers, subscripts and superscripts are available with 's' and 'S': 1s ₁ 1S ¹ ~ 2s ₂ 9S ⁹ ~ -But some don´t come defined by default. Those are digraph definitions you can +But some don't come defined by default. Those are digraph definitions you can add in your ~/.vimrc file. > exec 'digraph \\ '.char2nr('∖') exec 'digraph \< '.char2nr('≼') diff --git a/runtime/doc/help.txt b/runtime/doc/help.txt index 353058ec03..6416f49061 100644 --- a/runtime/doc/help.txt +++ b/runtime/doc/help.txt @@ -125,7 +125,7 @@ Advanced editing ~ |windows.txt| commands for using multiple windows and buffers |tabpage.txt| commands for using multiple tab pages |spell.txt| spell checking -|diff.txt| working with two to four versions of the same file +|diff.txt| working with two to eight versions of the same file |autocmd.txt| automatically executing commands on an event |eval.txt| expression evaluation, conditional commands |fold.txt| hide (fold) ranges of lines @@ -145,6 +145,8 @@ Programming language support ~ |filetype.txt| settings done specifically for a type of file |quickfix.txt| commands for a quick edit-compile-fix cycle |ft_ada.txt| Ada (the programming language) support +|ft_ps1.txt| Filetype plugin for Windows PowerShell +|ft_raku.txt| Filetype plugin for Raku |ft_rust.txt| Filetype plugin for Rust |ft_sql.txt| about the SQL filetype plugin diff --git a/runtime/doc/lua.txt b/runtime/doc/lua.txt index a44b2e42f5..3fd3875557 100644 --- a/runtime/doc/lua.txt +++ b/runtime/doc/lua.txt @@ -576,11 +576,11 @@ If you want to exclude visual selections from highlighting on yank, use vim.highlight.on_yank({opts}) *vim.highlight.on_yank()* Highlights the yanked text. The fields of the optional dict {opts} control the highlight: - - {higroup} highlight group for yanked region (default `"IncSearch"`) + - {higroup} highlight group for yanked region (default |hl-IncSearch|) - {timeout} time in ms before highlight is cleared (default `150`) - {on_macro} highlight when executing macro (default `false`) - {on_visual} highlight when yanking visual selection (default `true`) - - {event} event structure (default `vim.v.event`) + - {event} event structure (default |v:event|) vim.highlight.range({bufnr}, {ns}, {higroup}, {start}, {finish}, {rtype}, {inclusive}) *vim.highlight.range()* @@ -687,6 +687,19 @@ vim.diff({a}, {b}, {opts}) *vim.diff()* See {opts.result_type}. nil if {opts.on_hunk} is given. ------------------------------------------------------------------------------ +VIM.MPACK *lua-mpack* + +The *vim.mpack* module provides packing and unpacking of lua objects to +msgpack encoded strings. |vim.NIL| and |vim.empty_dict()| are supported. + +vim.mpack.pack({obj}) *vim.mpack.pack* + Packs a lua object {obj} and returns the msgpack representation as + a string + +vim.mpack.unpack({str}) *vim.mpack.unpack* + Unpacks the msgpack encoded {str} and returns a lua object + +------------------------------------------------------------------------------ VIM *lua-builtin* vim.api.{func}({...}) *vim.api* @@ -1243,37 +1256,6 @@ region({bufnr}, {pos1}, {pos2}, {regtype}, {inclusive}) *vim.region()* Return: ~ region lua table of the form {linenr = {startcol,endcol}} - *vim.register_keystroke_callback()* -register_keystroke_callback({fn}, {ns_id}) - Register a lua {fn} with an {id} to be run after every - keystroke. - - If {fn} is nil, it removes the callback for the associated - {ns_id} - Note: - {fn} will not be cleared from |nvim_buf_clear_namespace()| - - Note: - {fn} will receive the keystrokes after mappings have been - evaluated - - Parameters: ~ - {fn} function: Function to call. It should take one - argument, which is a string. The string will contain - the literal keys typed. See |i_CTRL-V| - {ns_id} number? Namespace ID. If not passed or 0, will - generate and return a new namespace ID from - |nvim_create_namesapce()| - - Return: ~ - number Namespace ID associated with {fn} - - Note: - {fn} will be automatically removed if an error occurs - while calling. This is to prevent the annoying situation - of every keystroke erroring while trying to remove a - broken callback. - schedule_wrap({cb}) *vim.schedule_wrap()* Defers callback `cb` until the Nvim API is safe to call. diff --git a/runtime/doc/motion.txt b/runtime/doc/motion.txt index 9f8acff88a..c473244827 100644 --- a/runtime/doc/motion.txt +++ b/runtime/doc/motion.txt @@ -52,9 +52,14 @@ or change text. The following operators are available: |<| < shift left |zf| zf define a fold |g@| g@ call function set with the 'operatorfunc' option - + *motion-count-multiplied* If the motion includes a count and the operator also had a count before it, the two counts are multiplied. For example: "2d3w" deletes six words. + *operator-doubled* +When doubling the operator it operates on a line. When using a count, before +or after the first character, that many lines are operated upon. Thus `3dd` +deletes three lines. A count before and after the first character is +multiplied, thus `2y3y` yanks six lines. After applying the operator the cursor is mostly left at the start of the text that was operated upon. For example, "yfe" doesn't move the cursor, but "yFe" @@ -187,9 +192,9 @@ l or *l* *$* *<End>* *<kEnd>* $ or <End> To the end of the line. When a count is given also go [count - 1] lines downward, or as far is possible. - |inclusive| motion. If a count of 2 of larger is + |inclusive| motion. If a count of 2 or larger is given and the cursor is on the last line, that is an - error an the cursor doesn't move. + error and the cursor doesn't move. In Visual mode the cursor goes to just after the last character in the line. When 'virtualedit' is active, "$" may move the cursor diff --git a/runtime/doc/options.txt b/runtime/doc/options.txt index d7bd91ad4f..90ba3b86d9 100644 --- a/runtime/doc/options.txt +++ b/runtime/doc/options.txt @@ -4352,19 +4352,21 @@ A jump table for the options with a short description can be found at |Q_op|. - abbreviations are disabled - 'autoindent' is reset - 'expandtab' is reset - - 'formatoptions' is used like it is empty + - 'hkmap' is reset - 'revins' is reset - 'ruler' is reset - 'showmatch' is reset - - 'smartindent' is reset - 'smarttab' is reset - 'softtabstop' is set to 0 - 'textwidth' is set to 0 - 'wrapmargin' is set to 0 + - 'varsofttabstop' is made empty These options keep their value, but their effect is disabled: - 'cindent' + - 'formatoptions' is used like it is empty - 'indentexpr' - 'lisp' + - 'smartindent' NOTE: When you start editing another file while the 'paste' option is on, settings from the modelines or autocommands may change the settings again, causing trouble when pasting text. You might want to diff --git a/runtime/doc/pattern.txt b/runtime/doc/pattern.txt index e74f3b72bf..c49cc6d540 100644 --- a/runtime/doc/pattern.txt +++ b/runtime/doc/pattern.txt @@ -1204,7 +1204,7 @@ x A single character, with no special meaning, matches itself \%d123 Matches the character specified with a decimal number. Must be followed by a non-digit. -\%o40 Matches the character specified with an octal number up to 0377. +\%o40 Matches the character specified with an octal number up to 0o377. Numbers below 0o40 must be followed by a non-octal digit or a non-digit. \%x2a Matches the character specified with up to two hexadecimal characters. diff --git a/runtime/doc/pi_netrw.txt b/runtime/doc/pi_netrw.txt index 4b61cd4c25..a39d1e8dc9 100644 --- a/runtime/doc/pi_netrw.txt +++ b/runtime/doc/pi_netrw.txt @@ -3808,7 +3808,7 @@ netrw: Decho.vim is provided as a "vimball"; see |vimball-intro|. You should edit the Decho.vba.gz file and source it in: > - vim Decho.vba.gz + vim Decho.vba.gz :so % :q < diff --git a/runtime/doc/repeat.txt b/runtime/doc/repeat.txt index a91aa4d2a0..ed770434d5 100644 --- a/runtime/doc/repeat.txt +++ b/runtime/doc/repeat.txt @@ -255,7 +255,9 @@ For writing a Vim script, see chapter 41 of the user manual |usr_41.txt|. ftdetect scripts are loaded, only the matching directories are added to 'runtimepath'. This is useful in your .vimrc. The plugins will then be - loaded during initialization, see |load-plugins|. + loaded during initialization, see |load-plugins| (note + that the loading order will be reversed, because each + directory is inserted before others). Note that for ftdetect scripts to be loaded you will need to write `filetype plugin indent on` AFTER all `packadd!` commands. diff --git a/runtime/doc/rileft.txt b/runtime/doc/rileft.txt index d45a2dce7e..aa11462595 100644 --- a/runtime/doc/rileft.txt +++ b/runtime/doc/rileft.txt @@ -70,7 +70,7 @@ o Invocations o Typing backwards *ins-reverse* ---------------- - In lieu of using full-fledged the 'rightleft' option, one can opt for + In lieu of using the full-fledged 'rightleft' option, one can opt for reverse insertion. When the 'revins' (reverse insert) option is set, inserting happens backwards. This can be used to type right-to-left text. When inserting characters the cursor is not moved and the text diff --git a/runtime/doc/starting.txt b/runtime/doc/starting.txt index d6b54fbd01..87a48e6d2a 100644 --- a/runtime/doc/starting.txt +++ b/runtime/doc/starting.txt @@ -329,12 +329,12 @@ argument. -w{number} Set the 'window' option to {number}. *-w* --w {scriptout} All the characters that you type are recorded in the file - "scriptout", until you exit Vim. This is useful if you want - to create a script file to be used with "vim -s" or - ":source!". When the "scriptout" file already exists, new - characters are appended. See also |complex-repeat|. - {scriptout} cannot start with a digit. +-w {scriptout} All keys that you type are recorded in the file "scriptout", + until you exit Vim. Useful to create a script file to be used + with "vim -s" or ":source!". Appends to the "scriptout" file + if it already exists. {scriptout} cannot start with a digit. + See also |vim.on_key()|. + See also |complex-repeat|. *-W* -W {scriptout} Like -w, but do not append, overwrite an existing file. diff --git a/runtime/doc/syntax.txt b/runtime/doc/syntax.txt index 161b4f0d04..a8d8d7d9b8 100644 --- a/runtime/doc/syntax.txt +++ b/runtime/doc/syntax.txt @@ -1384,11 +1384,11 @@ To select syntax highlighting file for Euphoria, as well as for auto-detecting the *.e and *.E file extensions as Euphoria file type, add the following line to your startup file: > - :let filetype_euphoria="euphoria3" + :let filetype_euphoria = "euphoria3" - or +< or > - :let filetype_euphoria="euphoria4" + :let filetype_euphoria = "euphoria4" ERLANG *erlang.vim* *ft-erlang-syntax* @@ -3385,8 +3385,8 @@ syntax highlighting script handles this with the following logic: Tex: Match Check Control~ Sometimes one actually wants mismatched parentheses, square braces, - and or curly braces; for example, \text{(1,10] is a range from but - not including 1 to and including 10}. This wish, of course, conflicts + and or curly braces; for example, \text{(1,10]} is a range from but + not including 1 to and including 10. This wish, of course, conflicts with the desire to provide delimiter mismatch detection. To accommodate these conflicting goals, syntax/tex.vim provides > g:tex_matchcheck = '[({[]' @@ -4024,7 +4024,7 @@ match in the same position overrules an earlier one). The "transparent" argument makes the "myVim" match use the same highlighting as "myString". But it does not contain anything. If the "contains=NONE" argument would be left out, then "myVim" would use the contains argument from myString and allow -"myWord" to be contained, which will be highlighted as a Constant. This +"myWord" to be contained, which will be highlighted as a Comment. This happens because a contained match doesn't match inside itself in the same position, thus the "myVim" match doesn't overrule the "myWord" match here. diff --git a/runtime/doc/treesitter.txt b/runtime/doc/treesitter.txt index a509c328e5..86316b8ac5 100644 --- a/runtime/doc/treesitter.txt +++ b/runtime/doc/treesitter.txt @@ -767,17 +767,4 @@ new({source}, {lang}, {opts}) *languagetree.new()* the injection language query per language. - -============================================================================== -Lua module: vim.treesitter.health *treesitter-health* - -check_health() *check_health()* - TODO: Documentation - -list_parsers() *list_parsers()* - Lists the parsers currently installed - - Return: ~ - A list of parsers - vim:tw=78:ts=8:ft=help:norl: diff --git a/runtime/doc/various.txt b/runtime/doc/various.txt index 62e1e130ee..29bd6e5e64 100644 --- a/runtime/doc/various.txt +++ b/runtime/doc/various.txt @@ -504,7 +504,7 @@ gO Show a filetype-specific, navigable "outline" of the Queued messages are processed during the sleep. *:sl!* *:sleep!* -:[N]sl[eep]! [N] [m] Same as above. Unlike Vim, it does not hide the +:[N]sl[eep]! [N][m] Same as above. Unlike Vim, it does not hide the cursor. |vim-differences| ============================================================================== diff --git a/runtime/doc/vim_diff.txt b/runtime/doc/vim_diff.txt index 4b134926a2..a5fcef2800 100644 --- a/runtime/doc/vim_diff.txt +++ b/runtime/doc/vim_diff.txt @@ -6,16 +6,16 @@ Differences between Nvim and Vim *vim-differences* -Nvim differs from Vim in many ways, although editor and VimL features are -mostly identical. This document is a complete and centralized reference of -the differences. +Nvim differs from Vim in many ways, although editor and Vimscript (not +Vim9script) features are mostly identical. This document is a complete and +centralized reference of the differences. Type |gO| to see the table of contents. ============================================================================== 1. Configuration *nvim-config* -- Use `$XDG_CONFIG_HOME/nvim/init.vim` instead of `.vimrc` for configuration. +- Use `$XDG_CONFIG_HOME/nvim/init.vim` instead of `.vimrc` for your |config|. - Use `$XDG_CONFIG_HOME/nvim` instead of `.vim` to store configuration files. - Use `$XDG_DATA_HOME/nvim/shada/main.shada` instead of `.viminfo` for persistent session information. |shada| @@ -78,6 +78,8 @@ the differences. Default Mappings ~ *default-mappings* +Nvim creates the following default mappings at |startup|. You can disable any +of these in your config by simply removing the mapping, e.g. ":unmap Y". > nnoremap Y y$ nnoremap <C-L> <Cmd>nohlsearch<Bar>diffupdate<CR><C-L> @@ -101,17 +103,19 @@ nvim_cmdwin: MAJOR COMPONENTS ~ API |API| -Lua scripting |lua| Job control |job-control| -Remote plugins |remote-plugin| +LSP framework |lsp| +Lua scripting |lua| +Parsing engine |treesitter| Providers Clipboard |provider-clipboard| Node.js plugins |provider-nodejs| Python plugins |provider-python| Ruby plugins |provider-ruby| +Remote plugins |remote-plugin| Shared data |shada| -Embedded terminal |terminal| -VimL parser |nvim_parse_expression()| +Terminal emulator |terminal| +Vimscript parser |nvim_parse_expression()| XDG base directories |xdg| USER EXPERIENCE ~ @@ -160,7 +164,7 @@ FEATURES ~ Command-line highlighting: The expression prompt (|@=|, |c_CTRL-R_=|, |i_CTRL-R_=|) is highlighted - using a built-in VimL expression parser. |expr-highlight| + using a built-in Vimscript expression parser. |expr-highlight| *E5408* *E5409* |input()|, |inputdialog()| support custom highlighting. |input()-highlight| *g:Nvim_color_cmdline* @@ -413,7 +417,7 @@ TUI: UI/Display: |Visual| selection highlights the character at cursor. |visual-use| -VimL (Vim script) compatibility: +Vimscript compatibility: `count` does not alias to |v:count| `errmsg` does not alias to |v:errmsg| `shell_error` does not alias to |v:shell_error| diff --git a/runtime/filetype.vim b/runtime/filetype.vim index 333da58128..b3148bde25 100644 --- a/runtime/filetype.vim +++ b/runtime/filetype.vim @@ -1,7 +1,7 @@ " Vim support file to detect file types " " Maintainer: Bram Moolenaar <Bram@vim.org> -" Last Change: 2021 Apr 17 +" Last Change: 2021 Jun 13 " Listen very carefully, I will say this only once if exists("did_load_filetypes") diff --git a/runtime/ftplugin/eruby.vim b/runtime/ftplugin/eruby.vim index 3c18bada78..e67b00b278 100644 --- a/runtime/ftplugin/eruby.vim +++ b/runtime/ftplugin/eruby.vim @@ -3,7 +3,7 @@ " Maintainer: Tim Pope <vimNOSPAM@tpope.org> " URL: https://github.com/vim-ruby/vim-ruby " Release Coordinator: Doug Kearns <dougkearns@gmail.com> -" Last Change: 2019 Jan 06 +" Last Change: 2020 Jun 28 " Only do this when not done yet for this buffer if exists("b:did_ftplugin") @@ -118,7 +118,7 @@ endif " TODO: comments= setlocal commentstring=<%#%s%> -let b:undo_ftplugin = "setl cms< " +let b:undo_ftplugin = "setl cms< " . \ " | unlet! b:browsefilter b:match_words | " . s:undo_ftplugin let &cpo = s:save_cpo diff --git a/runtime/ftplugin/ruby.vim b/runtime/ftplugin/ruby.vim index b4a8eaa0d8..4a476fd8cf 100644 --- a/runtime/ftplugin/ruby.vim +++ b/runtime/ftplugin/ruby.vim @@ -3,7 +3,7 @@ " Maintainer: Tim Pope <vimNOSPAM@tpope.org> " URL: https://github.com/vim-ruby/vim-ruby " Release Coordinator: Doug Kearns <dougkearns@gmail.com> -" Last Change: 2019 Nov 06 +" Last Change: 2020 Feb 13 if (exists("b:did_ftplugin")) finish @@ -112,7 +112,7 @@ else if !exists('g:ruby_default_path') if has("ruby") && has("win32") ruby ::VIM::command( 'let g:ruby_default_path = split("%s",",")' % $:.join(%q{,}) ) - elseif executable('ruby') + elseif executable('ruby') && !empty($HOME) let g:ruby_default_path = s:query_path($HOME) else let g:ruby_default_path = map(split($RUBYLIB,':'), 'v:val ==# "." ? "" : v:val') diff --git a/runtime/indent/html.vim b/runtime/indent/html.vim index 7019bd4a82..d4b91f6421 100644 --- a/runtime/indent/html.vim +++ b/runtime/indent/html.vim @@ -1,7 +1,7 @@ " Vim indent script for HTML " Maintainer: Bram Moolenaar " Original Author: Andy Wokula <anwoku@yahoo.de> -" Last Change: 2021 Jan 26 +" Last Change: 2021 Jun 13 " Version: 1.0 "{{{ " Description: HTML indent script with cached state for faster indenting on a " range of lines. @@ -62,7 +62,7 @@ let s:tagname = '\w\+\(-\w\+\)*' " Prefer using buffer-local settings over global settings, so that there can " be defaults for all HTML files and exceptions for specific types of HTML " files. -func! HtmlIndent_CheckUserSettings() +func HtmlIndent_CheckUserSettings() "{{{ let inctags = '' if exists("b:html_indent_inctags") @@ -178,7 +178,7 @@ let s:endtags = [0,0,0,0,0,0,0] " long enough for the highest index "}}} " Add a list of tag names for a pair of <tag> </tag> to "tags". -func! s:AddITags(tags, taglist) +func s:AddITags(tags, taglist) "{{{ for itag in a:taglist let a:tags[itag] = 1 @@ -187,7 +187,7 @@ func! s:AddITags(tags, taglist) endfunc "}}} " Take a list of tag name pairs that are not to be used as tag pairs. -func! s:RemoveITags(tags, taglist) +func s:RemoveITags(tags, taglist) "{{{ for itag in a:taglist let a:tags[itag] = 1 @@ -196,7 +196,7 @@ func! s:RemoveITags(tags, taglist) endfunc "}}} " Add a block tag, that is a tag with a different kind of indenting. -func! s:AddBlockTag(tag, id, ...) +func s:AddBlockTag(tag, id, ...) "{{{ if !(a:id >= 2 && a:id < len(s:endtags)) echoerr 'AddBlockTag ' . a:id @@ -255,7 +255,7 @@ call s:AddBlockTag('<!--[', 6, '![endif]-->') " Return non-zero when "tagname" is an opening tag, not being a block tag, for " which there should be a closing tag. Can be used by scripts that include " HTML indenting. -func! HtmlIndent_IsOpenTag(tagname) +func HtmlIndent_IsOpenTag(tagname) "{{{ if get(s:indent_tags, a:tagname) == 1 return 1 @@ -264,7 +264,7 @@ func! HtmlIndent_IsOpenTag(tagname) endfunc "}}} " Get the value for "tagname", taking care of buffer-local tags. -func! s:get_tag(tagname) +func s:get_tag(tagname) "{{{ let i = get(s:indent_tags, a:tagname) if (i == 1 || i == -1) && get(b:hi_removed_tags, a:tagname) != 0 @@ -277,7 +277,7 @@ func! s:get_tag(tagname) endfunc "}}} " Count the number of start and end tags in "text". -func! s:CountITags(text) +func s:CountITags(text) "{{{ " Store the result in s:curind and s:nextrel. let s:curind = 0 " relative indent steps for current line [unit &sw]: @@ -289,7 +289,7 @@ func! s:CountITags(text) endfunc "}}} " Count the number of start and end tags in text. -func! s:CountTagsAndState(text) +func s:CountTagsAndState(text) "{{{ " Store the result in s:curind and s:nextrel. Update b:hi_newstate.block. let s:curind = 0 " relative indent steps for current line [unit &sw]: @@ -304,7 +304,7 @@ func! s:CountTagsAndState(text) endfunc "}}} " Used by s:CountITags() and s:CountTagsAndState(). -func! s:CheckTag(itag) +func s:CheckTag(itag) "{{{ " Returns an empty string or "SCRIPT". " a:itag can be "tag" or "/tag" or "<!--" or "-->" @@ -338,7 +338,7 @@ func! s:CheckTag(itag) endfunc "}}} " Used by s:CheckTag(). Returns an empty string or "SCRIPT". -func! s:CheckBlockTag(blocktag, ind) +func s:CheckBlockTag(blocktag, ind) "{{{ if a:ind > 0 " a block starts here @@ -366,7 +366,7 @@ func! s:CheckBlockTag(blocktag, ind) endfunc "}}} " Used by s:CheckTag(). -func! s:CheckCustomTag(ctag) +func s:CheckCustomTag(ctag) "{{{ " Returns 1 if ctag is the tag for a custom element, 0 otherwise. " a:ctag can be "tag" or "/tag" or "<!--" or "-->" @@ -396,7 +396,7 @@ func! s:CheckCustomTag(ctag) endfunc "}}} " Return the <script> type: either "javascript" or "" -func! s:GetScriptType(str) +func s:GetScriptType(str) "{{{ if a:str == "" || a:str =~ "java" return "javascript" @@ -407,7 +407,7 @@ endfunc "}}} " Look back in the file, starting at a:lnum - 1, to compute a state for the " start of line a:lnum. Return the new state. -func! s:FreshState(lnum) +func s:FreshState(lnum) "{{{ " A state is to know ALL relevant details about the " lines 1..a:lnum-1, initial calculating (here!) can be slow, but updating is @@ -568,24 +568,29 @@ func! s:FreshState(lnum) endfunc "}}} " Indent inside a <pre> block: Keep indent as-is. -func! s:Alien2() +func s:Alien2() "{{{ return -1 endfunc "}}} " Return the indent inside a <script> block for javascript. -func! s:Alien3() +func s:Alien3() "{{{ let lnum = prevnonblank(v:lnum - 1) while lnum > 1 && getline(lnum) =~ '^\s*/[/*]' " Skip over comments to avoid that cindent() aligns with the <script> tag let lnum = prevnonblank(lnum - 1) endwhile + if lnum < b:hi_indent.blocklnr + " indent for <script> itself + return b:hi_indent.blocktagind + endif if lnum == b:hi_indent.blocklnr " indent for the first line after <script> return eval(b:hi_js1indent) endif if b:hi_indent.scripttype == "javascript" + " indent for further lines return eval(b:hi_js1indent) + GetJavascriptIndent() else return -1 @@ -593,7 +598,7 @@ func! s:Alien3() endfunc "}}} " Return the indent inside a <style> block. -func! s:Alien4() +func s:Alien4() "{{{ if prevnonblank(v:lnum-1) == b:hi_indent.blocklnr " indent for first content line @@ -603,7 +608,7 @@ func! s:Alien4() endfunc "}}} " Indending inside a <style> block. Returns the indent. -func! s:CSSIndent() +func s:CSSIndent() "{{{ " This handles standard CSS and also Closure stylesheets where special lines " start with @. @@ -720,13 +725,13 @@ endfunc "}}} " tag: blah " tag: blah && " tag: blah || -func! s:CssUnfinished(text) +func s:CssUnfinished(text) "{{{ return a:text =~ '\(||\|&&\|:\|\k\)\s*$' endfunc "}}} " Search back for the first unfinished line above "lnum". -func! s:CssFirstUnfinished(lnum, min_lnum) +func s:CssFirstUnfinished(lnum, min_lnum) "{{{ let align_lnum = a:lnum while align_lnum > a:min_lnum && s:CssUnfinished(getline(align_lnum - 1)) @@ -736,7 +741,7 @@ func! s:CssFirstUnfinished(lnum, min_lnum) endfunc "}}} " Find the non-empty line at or before "lnum" that is not a comment. -func! s:CssPrevNonComment(lnum, stopline) +func s:CssPrevNonComment(lnum, stopline) "{{{ " caller starts from a line a:lnum + 1 that is not a comment let lnum = prevnonblank(a:lnum) @@ -761,7 +766,7 @@ func! s:CssPrevNonComment(lnum, stopline) endfunc "}}} " Check the number of {} and () in line "lnum". Return a dict with the counts. -func! HtmlIndent_CountBraces(lnum) +func HtmlIndent_CountBraces(lnum) "{{{ let brs = substitute(getline(a:lnum), '[''"].\{-}[''"]\|/\*.\{-}\*/\|/\*.*$\|[^{}()]', '', 'g') let c_open = 0 @@ -794,7 +799,7 @@ func! HtmlIndent_CountBraces(lnum) endfunc "}}} " Return the indent for a comment: <!-- --> -func! s:Alien5() +func s:Alien5() "{{{ let curtext = getline(v:lnum) if curtext =~ '^\s*\zs-->' @@ -826,7 +831,7 @@ func! s:Alien5() endfunc "}}} " Return the indent for conditional comment: <!--[ ![endif]--> -func! s:Alien6() +func s:Alien6() "{{{ let curtext = getline(v:lnum) if curtext =~ '\s*\zs<!\[endif\]-->' @@ -840,7 +845,7 @@ func! s:Alien6() endfunc "}}} " When the "lnum" line ends in ">" find the line containing the matching "<". -func! HtmlIndent_FindTagStart(lnum) +func HtmlIndent_FindTagStart(lnum) "{{{ " Avoids using the indent of a continuation line. " Moves the cursor. @@ -863,7 +868,7 @@ func! HtmlIndent_FindTagStart(lnum) endfunc "}}} " Find the unclosed start tag from the current cursor position. -func! HtmlIndent_FindStartTag() +func HtmlIndent_FindStartTag() "{{{ " The cursor must be on or before a closing tag. " If found, positions the cursor at the match and returns the line number. @@ -877,7 +882,7 @@ func! HtmlIndent_FindStartTag() endfunc "}}} " Moves the cursor from a "<" to the matching ">". -func! HtmlIndent_FindTagEnd() +func HtmlIndent_FindTagEnd() "{{{ " Call this with the cursor on the "<" of a start tag. " This will move the cursor to the ">" of the matching end tag or, when it's @@ -897,7 +902,7 @@ func! HtmlIndent_FindTagEnd() endfunc "}}} " Indenting inside a start tag. Return the correct indent or -1 if unknown. -func! s:InsideTag(foundHtmlString) +func s:InsideTag(foundHtmlString) "{{{ if a:foundHtmlString " Inside an attribute string. @@ -958,7 +963,7 @@ func! s:InsideTag(foundHtmlString) endfunc "}}} " THE MAIN INDENT FUNCTION. Return the amount of indent for v:lnum. -func! HtmlIndent() +func HtmlIndent() "{{{ if prevnonblank(v:lnum - 1) < 1 " First non-blank line has no indent. diff --git a/runtime/indent/ruby.vim b/runtime/indent/ruby.vim index 657aa763b1..2a267fdab3 100644 --- a/runtime/indent/ruby.vim +++ b/runtime/indent/ruby.vim @@ -4,6 +4,7 @@ " Previous Maintainer: Nikolai Weibull <now at bitwi.se> " URL: https://github.com/vim-ruby/vim-ruby " Release Coordinator: Doug Kearns <dougkearns@gmail.com> +" Last Change: 2021 Feb 03 " 0. Initialization {{{1 " ================= diff --git a/runtime/syntax/8th.vim b/runtime/syntax/8th.vim index ddc1084c9f..d543489b72 100644 --- a/runtime/syntax/8th.vim +++ b/runtime/syntax/8th.vim @@ -293,7 +293,7 @@ syn region eighthComment start="\zs\\" end="$" contains=eighthTodo " Define the default highlighting. if !exists("did_eighth_syntax_inits") let did_eighth_syntax_inits=1 - " The default methods for highlighting. Can be overriden later. + " The default methods for highlighting. Can be overridden later. hi def link eighthTodo Todo hi def link eighthOperators Operator hi def link eighthMath Number diff --git a/runtime/syntax/gvpr.vim b/runtime/syntax/gvpr.vim new file mode 100644 index 0000000000..a7378916f9 --- /dev/null +++ b/runtime/syntax/gvpr.vim @@ -0,0 +1,85 @@ +" Vim syntax file +" Language: Graphviz program +" Maintainer: Matthew Fernandez <matthew.fernandez@gmail.com> +" Last Change: Tue, 28 Jul 2020 17:20:44 -0700 + +if exists("b:current_syntax") + finish +endif + +let s:cpo_save = &cpo +set cpo&vim + +syn keyword gvArg ARGC ARGV +syn keyword gvBeg BEGIN BEG_G N E END END_G +syn keyword gvFunc + \ graph fstsubg isDirect isStrict isSubg nEdges nNodes nxtsubg subg + \ degreeOf fstnode indegreeOf isNode isSubnode node nxtnode nxtnode_sg + \ outDegreeOf subnode + \ edge edge_sg fstedge fstedge_sg fstin fstin_sg fstout fstout_sg isEdge + \ isEdge_sg isSubedge nxtedge nxtedge_sg nxtin nxtin_sg nxtout nxtout_sg opp + \ subedge + \ freadG fwriteG readG write[] writeG + \ aget aset clone cloneG compOf copy[] copyA delete[] fstAttr getDflt hasAttr + \ induce isAttr isIn kindOf lock[] nxtAttr setDflt + \ canon gsub html index ishtml length llOf match[] rindex split[] sprintf + \ sscanf strcmp sub substr tokens tolower toupper urOf xOf yOf + \ closeF openF print[] printf scanf readL + \ atan2 cos exp log MAX MIN pow sin[] sqrt + \ in[] unset + \ colorx exit[] rand srand system +syn keyword gvCons + \ NULL TV_bfs TV_dfs TV_en TV_flat TV_fwd TV_ne TV_prepostdfs TV_prepostfwd + \ TV_prepostrev TV_postdfs TV_postfwd tv_postrev TV_rev +syn keyword gvType char double float int long unsigned void + \ string + \ edge_t graph_t node_t obj_t +syn match gvVar + \ "\$\(\(F\|G\|NG\|O\|T\|tgtname\|tvedge\|tvnext\|tvroot\|tvtype\)\>\)\?\(\<\)\@!" +syn keyword gvWord break continue else for forr if return switch while + +" numbers adapted from c.vim's cNumbers and friends +syn match gvNums transparent "\<\d\|\.\d" contains=gvNumber,gvFloat,gvOctal +syn match gvNumber contained "\d\+\(u\=l\{0,2}\|ll\=u\)\>" +syn match gvNumber contained "0x\x\+\(u\=l\{0,2}\|ll\=u\)\>" +syn match gvOctal contained "0\o\+\(u\=l\{0,2}\|ll\=u\)\>" contains=gvOctalZero +syn match gvOctalZero contained "\<0" +syn match gvFloat contained "\d\+f" +syn match gvFloat contained "\d\+\.\d*\(e[-+]\=\d\+\)\=[fl]\=" +syn match gvFloat contained "\.\d\+\(e[-+]\=\d\+\)\=[fl]\=\>" +syn match gvFloat contained "\d\+e[-+]\=\d\+[fl]\=\>" + +syn region gvString start=+"+ skip=+\\\\\|\\"+ end=+"+ contains=gvFormat,gvSpecial extend +syn region gvString start="'" skip="\\\\\|\\'" end="'" contains=gvFormat,gvSpecial extend + +" adapted from c.vim's cFormat for c_no_c99 +syn match gvFormat "%\(\d\+\$\)\=[-+' #0*]*\(\d*\|\*\|\*\d\+\$\)\(\.\(\d*\|\*\|\*\d\+\$\)\)\=\([hlL]\|ll\)\=\([bdiuoxXDOUfeEgGcCsSpn]\|\[\^\=.[^]]*\]\)" contained + +syn match gvSpecial "\\." contained + +syn region gvCComment start="//" skip="\\$" end="$" keepend +syn region gvCPPComment start="#" skip="\\$" end="$" keepend +syn region gvCXXComment start="/\*" end="\*/" fold + +hi def link gvArg Identifier +hi def link gvBeg Keyword +hi def link gvFloat Number +hi def link gvFunc Identifier +hi def link gvCons Number +hi def link gvNumber Number +hi def link gvType Type +hi def link gvVar Statement +hi def link gvWord Keyword + +hi def link gvString String +hi def link gvFormat Special +hi def link gvSpecial Special + +hi def link gvCComment Comment +hi def link gvCPPComment Comment +hi def link gvCXXComment Comment + +let b:current_syntax = "gvpr" + +let &cpo = s:cpo_save +unlet s:cpo_save diff --git a/runtime/syntax/help.vim b/runtime/syntax/help.vim index d3d8f4f435..01915d23d7 100644 --- a/runtime/syntax/help.vim +++ b/runtime/syntax/help.vim @@ -1,7 +1,7 @@ " Vim syntax file " Language: Vim help file " Maintainer: Bram Moolenaar (Bram@vim.org) -" Last Change: 2020 Jul 28 +" Last Change: 2021 Jun 13 " Quit when a (custom) syntax file was already loaded if exists("b:current_syntax") diff --git a/runtime/syntax/redif.vim b/runtime/syntax/redif.vim index 725067fd32..198d5c7530 100644 --- a/runtime/syntax/redif.vim +++ b/runtime/syntax/redif.vim @@ -1,7 +1,7 @@ " Vim syntax file " Language: ReDIF " Maintainer: Axel Castellane <axel.castellane@polytechnique.edu> -" Last Change: 2013 April 17 +" Last Change: 2021 Jun 17 " Original Author: Axel Castellane " Source: http://openlib.org/acmes/root/docu/redif_1.html " File Extension: rdf diff --git a/runtime/syntax/ruby.vim b/runtime/syntax/ruby.vim index 0de63d0ef3..13d6d9efd8 100644 --- a/runtime/syntax/ruby.vim +++ b/runtime/syntax/ruby.vim @@ -3,7 +3,7 @@ " Maintainer: Doug Kearns <dougkearns@gmail.com> " URL: https://github.com/vim-ruby/vim-ruby " Release Coordinator: Doug Kearns <dougkearns@gmail.com> -" Last Change: 2019 Jul 13 +" Last Change: 2021 Jun 06 " ---------------------------------------------------------------------------- " " Previous Maintainer: Mirko Nasato @@ -66,7 +66,7 @@ endfunction com! -nargs=* SynFold call s:run_syntax_fold(<q-args>) " Not-Top Cluster {{{1 -syn cluster rubyNotTop contains=@rubyCommentNotTop,@rubyStringNotTop,@rubyRegexpSpecial,@rubyDeclaration,@rubyExceptionHandler,@rubyClassOperator,rubyConditional,rubyModuleName,rubyClassName,rubySymbolDelimiter,rubyParentheses +syn cluster rubyNotTop contains=@rubyCommentNotTop,@rubyStringNotTop,@rubyRegexpSpecial,@rubyDeclaration,@rubyExceptionHandler,@rubyClassOperator,rubyConditional,rubyModuleName,rubyClassName,rubySymbolDelimiter,rubyParentheses,@Spell " Whitespace Errors {{{1 if exists("ruby_space_errors") @@ -92,7 +92,7 @@ if exists("ruby_operators") || exists("ruby_pseudo_operators") syn match rubyBooleanOperator "\%(\w\|[^\x00-\x7F]\)\@1<!!\|&&\|||" syn match rubyRangeOperator "\.\.\.\=" syn match rubyAssignmentOperator "=>\@!\|-=\|/=\|\*\*=\|\*=\|&&=\|&=\|||=\||=\|%=\|+=\|>>=\|<<=\|\^=" - syn match rubyAssignmentOperator "=>\@!" containedin=rubyBlockParameterList " TODO: this is inelegant + syn match rubyAssignmentOperator "=>\@!" contained containedin=rubyBlockParameterList " TODO: this is inelegant syn match rubyEqualityOperator "===\|==\|!=\|!\~\|=\~" syn region rubyBracketOperator matchgroup=rubyOperator start="\%(\%(\w\|[^\x00-\x7F]\)[?!]\=\|[]})]\)\@2<=\[" end="]" contains=ALLBUT,@rubyNotTop @@ -134,10 +134,10 @@ syn match rubyCurlyBraceEscape "\\[{}]" contained display syn match rubyAngleBracketEscape "\\[<>]" contained display syn match rubySquareBracketEscape "\\[[\]]" contained display -syn region rubyNestedParentheses start="(" skip="\\\\\|\\)" matchgroup=rubyString end=")" transparent contained -syn region rubyNestedCurlyBraces start="{" skip="\\\\\|\\}" matchgroup=rubyString end="}" transparent contained -syn region rubyNestedAngleBrackets start="<" skip="\\\\\|\\>" matchgroup=rubyString end=">" transparent contained -syn region rubyNestedSquareBrackets start="\[" skip="\\\\\|\\\]" matchgroup=rubyString end="\]" transparent contained +syn region rubyNestedParentheses start="(" skip="\\\\\|\\)" end=")" transparent contained +syn region rubyNestedCurlyBraces start="{" skip="\\\\\|\\}" end="}" transparent contained +syn region rubyNestedAngleBrackets start="<" skip="\\\\\|\\>" end=">" transparent contained +syn region rubyNestedSquareBrackets start="\[" skip="\\\\\|\\\]" end="\]" transparent contained syn cluster rubySingleCharEscape contains=rubyBackslashEscape,rubyQuoteEscape,rubySpaceEscape,rubyParenthesisEscape,rubyCurlyBraceEscape,rubyAngleBracketEscape,rubySquareBracketEscape syn cluster rubyNestedBrackets contains=rubyNested.\+ @@ -193,7 +193,7 @@ SynFold ':' syn region rubySymbol matchgroup=rubySymbolDelimiter start="[]})\"': syn match rubyCapitalizedMethod "\%(\%(^\|[^.]\)\.\s*\)\@<!\<\u\%(\w\|[^\x00-\x7F]\)*\>\%(\s*(\)\@=" -syn region rubyParentheses start="(" end=")" contains=ALLBUT,@rubyNotTop containedin=rubyBlockParameterList +syn region rubyParentheses start="(" end=")" contains=ALLBUT,@rubyNotTop contained containedin=rubyBlockParameterList syn region rubyBlockParameterList start="\%(\%(\<do\>\|{\)\_s*\)\@32<=|" end="|" contains=ALLBUT,@rubyNotTop,@rubyProperOperator if exists('ruby_global_variable_error') @@ -332,7 +332,7 @@ SynFold '<<' syn region rubyString start=+\%(\%(class\|::\|\.\@1<!\.\)\_s*\|\%([ syn match rubyAliasDeclaration "[^[:space:];#.()]\+" contained contains=rubySymbol,@rubyGlobalVariable nextgroup=rubyAliasDeclaration2 skipwhite syn match rubyAliasDeclaration2 "[^[:space:];#.()]\+" contained contains=rubySymbol,@rubyGlobalVariable syn match rubyMethodDeclaration "[^[:space:];#(]\+" contained contains=rubyConstant,rubyBoolean,rubyPseudoVariable,rubyInstanceVariable,rubyClassVariable,rubyGlobalVariable -syn match rubyClassDeclaration "[^[:space:];#<]\+" contained contains=rubyClassName,rubyScopeOperator nextgroup=rubySuperClassOperator skipwhite skipnl +syn match rubyClassDeclaration "[^[:space:];#<]\+" contained contains=rubyClassName,rubyScopeOperator nextgroup=rubySuperClassOperator skipwhite syn match rubyModuleDeclaration "[^[:space:];#<]\+" contained contains=rubyModuleName,rubyScopeOperator syn match rubyMethodName "\<\%([_[:alpha:]]\|[^\x00-\x7F]\)\%([_[:alnum:]]\|[^\x00-\x7F]\)*[?!=]\=\%([[:alnum:]_.:?!=]\|[^\x00-\x7F]\)\@!" contained containedin=rubyMethodDeclaration @@ -462,7 +462,7 @@ endif syn match rubyDefinedOperator "\%#=1\<defined?" display " 1.9-style Hash Keys and Keyword Parameters {{{1 -syn match rubySymbol "\%([{(|,]\_s*\)\@<=\%(\h\|[^\x00-\x7F]\)\%(\w\|[^\x00-\x7F]\)*[?!]\=::\@!"he=e-1 +syn match rubySymbol "\%(\w\|[^\x00-\x7F]\)\%(\w\|[^\x00-\x7F]\)*[?!]\=::\@!"he=e-1 contained containedin=rubyBlockParameterList,rubyCurlyBlock syn match rubySymbol "[]})\"':]\@1<!\<\%(\h\|[^\x00-\x7F]\)\%(\w\|[^\x00-\x7F]\)*[!?]\=:[[:space:],;]\@="he=e-1 syn match rubySymbol "[[:space:],{(]\%(\h\|[^\x00-\x7F]\)\%(\w\|[^\x00-\x7F]\)*[!?]\=:[[:space:],;]\@="hs=s+1,he=e-1 diff --git a/runtime/syntax/vim.vim b/runtime/syntax/vim.vim index 55c47aa34d..7aae7965a9 100644 --- a/runtime/syntax/vim.vim +++ b/runtime/syntax/vim.vim @@ -200,7 +200,7 @@ syn keyword vimAugroupKey contained aug[roup] " Operators: {{{2 " ========= -syn cluster vimOperGroup contains=vimEnvvar,vimFunc,vimFuncVar,vimOper,vimOperParen,vimNumber,vimString,vimRegister,vimContinue,vim9Comment +syn cluster vimOperGroup contains=vimEnvvar,vimFunc,vimFuncVar,vimOper,vimOperParen,vimNumber,vimString,vimType,vimRegister,vimContinue,vim9Comment syn match vimOper "\%#=1\(==\|!=\|>=\|<=\|=\~\|!\~\|>\|<\|=\)[?#]\{0,2}" skipwhite nextgroup=vimString,vimSpecFile syn match vimOper "\(\<is\|\<isnot\)[?#]\{0,2}\>" skipwhite nextgroup=vimString,vimSpecFile syn match vimOper "||\|&&\|[-+.!]" skipwhite nextgroup=vimString,vimSpecFile @@ -214,12 +214,13 @@ endif " ========= syn cluster vimFuncList contains=vimCommand,vimFunctionError,vimFuncKey,Tag,vimFuncSID syn cluster vimFuncBodyList contains=vimAbb,vimAddress,vimAugroupKey,vimAutoCmd,vimCmplxRepeat,vimComment,vim9Comment,vimContinue,vimCtrlChar,vimEcho,vimEchoHL,vimEnvvar,vimExecute,vimIsCommand,vimFBVar,vimFunc,vimFunction,vimFuncVar,vimGlobal,vimHighlight,vimIsCommand,vimLet,vimLetHereDoc,vimLineComment,vimMap,vimMark,vimNorm,vimNotation,vimNotFunc,vimNumber,vimOper,vimOperParen,vimRegion,vimRegister,vimSearch,vimSet,vimSpecFile,vimString,vimSubst,vimSynLine,vimUnmap,vimUserCommand -syn match vimFunction "\<fu\%[nction]!\=\s\+\%(<[sS][iI][dD]>\|[sSgGbBwWtTlL]:\)\=\%(\i\|[#.]\|{.\{-1,}}\)*\ze\s*(" contains=@vimFuncList nextgroup=vimFuncBody +syn match vimFunction "\<\(fu\%[nction]\)!\=\s\+\%(<[sS][iI][dD]>\|[sSgGbBwWtTlL]:\)\=\%(\i\|[#.]\|{.\{-1,}}\)*\ze\s*(" contains=@vimFuncList nextgroup=vimFuncBody + syn match vimFunction "\<def!\=\ze\s*(" contains=@vimFuncList nextgroup=vimFuncBody if exists("g:vimsyn_folding") && g:vimsyn_folding =~# 'f' syn region vimFuncBody contained fold start="\ze\s*(" matchgroup=vimCommand end="\<\(endf\>\|endfu\%[nction]\>\|enddef\>\)" contains=@vimFuncBodyList else - syn region vimFuncBody contained start="\ze\s*(" matchgroup=vimCommand end="\<\(endf\>\|endfu\%[nction]\>\|enddef\>\)" contains=@vimFuncBodyList + syn region vimFuncBody contained start="\ze\s*(" matchgroup=vimCommand end="\<\(endf\>\|endfu\%[nction]\>\|enddef\>\)" contains=@vimFuncBodyList endif syn match vimFuncVar contained "a:\(\K\k*\|\d\+\)" syn match vimFuncSID contained "\c<sid>\|\<s:" @@ -228,6 +229,9 @@ syn match vimFuncBlank contained "\s\+" syn keyword vimPattern contained start skip end +" vimTypes : new for vim9 + syn match vimType ":\s*\zs\<\(bool\|number\|float\|string\|blob\|list<\|dict<\|job\|channel\|func\)\>" + " Special Filenames, Modifiers, Extension Removal: {{{2 " =============================================== syn match vimSpecFile "<c\(word\|WORD\)>" nextgroup=vimSpecFileMod,vimSubst @@ -355,7 +359,7 @@ syn match vimCmplxRepeat '[^a-zA-Z_/\\()]q[0-9a-zA-Z"]\>'lc=1 syn match vimCmplxRepeat '@[0-9a-z".=@:]\ze\($\|[^a-zA-Z]\>\)' " Set command and associated set-options (vimOptions) with comment {{{2 -syn region vimSet matchgroup=vimCommand start="\<\%(setl\%[ocal]\|setg\%[lobal]\|se\%[t]\)\>" skip="\%(\\\\\)*\\." end="$" end="|" matchgroup=vimNotation end="<[cC][rR]>" oneline keepend contains=vimSetEqual,vimOption,vimErrSetting,vimComment,vim9Comment,vimSetString,vimSetMod +syn region vimSet matchgroup=vimCommand start="\<\%(setl\%[ocal]\|setg\%[lobal]\|se\%[t]\)\>" skip="\%(\\\\\)*\\.\n\@!" end="$" end="|" matchgroup=vimNotation end="<[cC][rR]>" keepend contains=vimSetEqual,vimOption,vimErrSetting,vimComment,vim9Comment,vimSetString,vimSetMod syn region vimSetEqual contained start="[=:]\|[-+^]=" skip="\\\\\|\\\s" end="[| \t]"me=e-1 end="$" contains=vimCtrlChar,vimSetSep,vimNotation,vimEnvvar syn region vimSetString contained start=+="+hs=s+1 skip=+\\\\\|\\"+ end=+"+ contains=vimCtrlChar syn match vimSetSep contained "[,:]" @@ -390,7 +394,7 @@ syn case match " Maps: {{{2 " ==== syn match vimMap "\<map\>!\=\ze\s*[^(]" skipwhite nextgroup=vimMapMod,vimMapLhs -syn keyword vimMap cm[ap] cno[remap] im[ap] ino[remap] lm[ap] ln[oremap] nm[ap] nn[oremap] no[remap] om[ap] ono[remap] smap snor[emap] vm[ap] vn[oremap] xm[ap] xn[oremap] skipwhite nextgroup=vimMapBang,vimMapMod,vimMapLhs +syn keyword vimMap cm[ap] cno[remap] im[ap] ino[remap] lm[ap] ln[oremap] nm[ap] nn[oremap] no[remap] om[ap] ono[remap] smap snor[emap] tno[remap] tm[ap] vm[ap] vmapc[lear] vn[oremap] xm[ap] xn[oremap] skipwhite nextgroup=vimMapBang,vimMapMod,vimMapLhs syn keyword nvimMap tn[oremap] tm[ap] skipwhite nextgroup=vimMapBang,vimMapMod,vimMapLhs syn keyword vimMap mapc[lear] smapc[lear] syn keyword vimUnmap cu[nmap] iu[nmap] lu[nmap] nun[map] ou[nmap] sunm[ap] unm[ap] unm[ap] vu[nmap] xu[nmap] skipwhite nextgroup=vimMapBang,vimMapMod,vimMapLhs @@ -981,6 +985,7 @@ if !exists("skip_vim_syntax_inits") hi def link vimSyntax vimCommand hi def link vimSynType vimSpecial hi def link vimTodo Todo + hi def link vimType Type hi def link vimUnmap vimMap hi def link vimUserAttrbCmpltFunc Special hi def link vimUserAttrbCmplt vimSpecial diff --git a/scripts/gen_vimdoc.py b/scripts/gen_vimdoc.py index 2efa544c2e..320c44e860 100755 --- a/scripts/gen_vimdoc.py +++ b/scripts/gen_vimdoc.py @@ -197,7 +197,6 @@ CONFIG = { 'query.lua', 'highlighter.lua', 'languagetree.lua', - 'health.lua', ], 'files': ' '.join([ os.path.join(base_dir, 'runtime/lua/vim/treesitter.lua'), @@ -1131,7 +1130,7 @@ Doxyfile = textwrap.dedent(''' INPUT_FILTER = "{filter}" EXCLUDE = EXCLUDE_SYMLINKS = NO - EXCLUDE_PATTERNS = */private/* + EXCLUDE_PATTERNS = */private/* */health.lua EXCLUDE_SYMBOLS = EXTENSION_MAPPING = lua=C EXTRACT_PRIVATE = NO diff --git a/src/mpack/LICENSE-MIT b/src/mpack/LICENSE-MIT new file mode 100644 index 0000000000..030ba872c5 --- /dev/null +++ b/src/mpack/LICENSE-MIT @@ -0,0 +1,22 @@ +Copyright (c) 2016 Thiago de Arruda + +Permission is hereby granted, free of charge, to any person +obtaining a copy of this software and associated documentation +files (the "Software"), to deal in the Software without +restriction, including without limitation the rights to use, +copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the +Software is furnished to do so, subject to the following +conditions: + +The above copyright notice and this permission notice shall be +included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES +OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT +HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, +WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR +OTHER DEALINGS IN THE SOFTWARE. diff --git a/src/mpack/conv.c b/src/mpack/conv.c new file mode 100644 index 0000000000..203b13fadb --- /dev/null +++ b/src/mpack/conv.c @@ -0,0 +1,375 @@ +#include "conv.h" + +static int mpack_fits_single(double v); +static mpack_value_t mpack_pack_ieee754(double v, unsigned m, unsigned e); +static int mpack_is_be(void) FPURE; +static double mpack_fmod_pow2_32(double a); + + +#define POW2(n) \ + ((double)(1 << (n / 2)) * (double)(1 << (n / 2)) * (double)(1 << (n % 2))) + +#define MPACK_SWAP_VALUE(val) \ + do { \ + mpack_uint32_t lo = val.lo; \ + val.lo = val.hi; \ + val.hi = lo; \ + } while (0) + +MPACK_API mpack_token_t mpack_pack_nil(void) +{ + mpack_token_t rv; + rv.type = MPACK_TOKEN_NIL; + return rv; +} + +MPACK_API mpack_token_t mpack_pack_boolean(unsigned v) +{ + mpack_token_t rv; + rv.type = MPACK_TOKEN_BOOLEAN; + rv.data.value.lo = v ? 1 : 0; + rv.data.value.hi = 0; + return rv; +} + +MPACK_API mpack_token_t mpack_pack_uint(mpack_uintmax_t v) +{ + mpack_token_t rv; + rv.data.value.lo = v & 0xffffffff; + rv.data.value.hi = (mpack_uint32_t)((v >> 31) >> 1); + rv.type = MPACK_TOKEN_UINT; + return rv; +} + +MPACK_API mpack_token_t mpack_pack_sint(mpack_sintmax_t v) +{ + if (v < 0) { + mpack_token_t rv; + mpack_uintmax_t tc = -((mpack_uintmax_t)(v + 1)) + 1; + tc = ~tc + 1; + rv = mpack_pack_uint(tc); + rv.type = MPACK_TOKEN_SINT; + return rv; + } + + return mpack_pack_uint((mpack_uintmax_t)v); +} + +MPACK_API mpack_token_t mpack_pack_float_compat(double v) +{ + /* ieee754 single-precision limits to determine if "v" can be fully + * represented in 4 bytes */ + mpack_token_t rv; + + if (mpack_fits_single(v)) { + rv.length = 4; + rv.data.value = mpack_pack_ieee754(v, 23, 8); + } else { + rv.length = 8; + rv.data.value = mpack_pack_ieee754(v, 52, 11); + } + + rv.type = MPACK_TOKEN_FLOAT; + return rv; +} + +MPACK_API mpack_token_t mpack_pack_float_fast(double v) +{ + /* ieee754 single-precision limits to determine if "v" can be fully + * represented in 4 bytes */ + mpack_token_t rv; + + if (mpack_fits_single(v)) { + union { + float f; + mpack_uint32_t m; + } conv; + conv.f = (float)v; + rv.length = 4; + rv.data.value.lo = conv.m; + rv.data.value.hi = 0; + } else { + union { + double d; + mpack_value_t m; + } conv; + conv.d = v; + rv.length = 8; + rv.data.value = conv.m; + if (mpack_is_be()) { + MPACK_SWAP_VALUE(rv.data.value); + } + } + + rv.type = MPACK_TOKEN_FLOAT; + return rv; +} + +MPACK_API mpack_token_t mpack_pack_number(double v) +{ + mpack_token_t tok; + double vabs; + vabs = v < 0 ? -v : v; + assert(v <= 9007199254740991. && v >= -9007199254740991.); + tok.data.value.hi = (mpack_uint32_t)(vabs / POW2(32)); + tok.data.value.lo = (mpack_uint32_t)mpack_fmod_pow2_32(vabs); + + if (v < 0) { + /* Compute the two's complement */ + tok.type = MPACK_TOKEN_SINT; + tok.data.value.hi = ~tok.data.value.hi; + tok.data.value.lo = ~tok.data.value.lo + 1; + if (!tok.data.value.lo) tok.data.value.hi++; + if (tok.data.value.lo == 0 && tok.data.value.hi == 0) tok.length = 1; + else if (tok.data.value.lo < 0x80000000) tok.length = 8; + else if (tok.data.value.lo < 0xffff7fff) tok.length = 4; + else if (tok.data.value.lo < 0xffffff7f) tok.length = 2; + else tok.length = 1; + } else { + tok.type = MPACK_TOKEN_UINT; + if (tok.data.value.hi) tok.length = 8; + else if (tok.data.value.lo > 0xffff) tok.length = 4; + else if (tok.data.value.lo > 0xff) tok.length = 2; + else tok.length = 1; + } + + if (mpack_unpack_number(tok) != v) { + return mpack_pack_float(v); + } + + return tok; +} + +MPACK_API mpack_token_t mpack_pack_chunk(const char *p, mpack_uint32_t l) +{ + mpack_token_t rv; + rv.type = MPACK_TOKEN_CHUNK; + rv.data.chunk_ptr = p; + rv.length = l; + return rv; +} + +MPACK_API mpack_token_t mpack_pack_str(mpack_uint32_t l) +{ + mpack_token_t rv; + rv.type = MPACK_TOKEN_STR; + rv.length = l; + return rv; +} + +MPACK_API mpack_token_t mpack_pack_bin(mpack_uint32_t l) +{ + mpack_token_t rv; + rv.type = MPACK_TOKEN_BIN; + rv.length = l; + return rv; +} + +MPACK_API mpack_token_t mpack_pack_ext(int t, mpack_uint32_t l) +{ + mpack_token_t rv; + rv.type = MPACK_TOKEN_EXT; + rv.length = l; + rv.data.ext_type = t; + return rv; +} + +MPACK_API mpack_token_t mpack_pack_array(mpack_uint32_t l) +{ + mpack_token_t rv; + rv.type = MPACK_TOKEN_ARRAY; + rv.length = l; + return rv; +} + +MPACK_API mpack_token_t mpack_pack_map(mpack_uint32_t l) +{ + mpack_token_t rv; + rv.type = MPACK_TOKEN_MAP; + rv.length = l; + return rv; +} + +MPACK_API bool mpack_unpack_boolean(mpack_token_t t) +{ + return t.data.value.lo || t.data.value.hi; +} + +MPACK_API mpack_uintmax_t mpack_unpack_uint(mpack_token_t t) +{ + return (((mpack_uintmax_t)t.data.value.hi << 31) << 1) | t.data.value.lo; +} + +/* unpack signed integer without relying on two's complement as internal + * representation */ +MPACK_API mpack_sintmax_t mpack_unpack_sint(mpack_token_t t) +{ + mpack_uint32_t hi = t.data.value.hi; + mpack_uint32_t lo = t.data.value.lo; + mpack_uintmax_t rv = lo; + assert(t.length <= sizeof(mpack_sintmax_t)); + + if (t.length == 8) { + rv |= (((mpack_uintmax_t)hi) << 31) << 1; + } + /* reverse the two's complement so that lo/hi contain the absolute value. + * note that we have to mask ~rv so that it reflects the two's complement + * of the appropriate byte length */ + rv = (~rv & (((mpack_uintmax_t)1 << ((t.length * 8) - 1)) - 1)) + 1; + /* negate and return the absolute value, making sure mpack_sintmax_t can + * represent the positive cast. */ + return -((mpack_sintmax_t)(rv - 1)) - 1; +} + +MPACK_API double mpack_unpack_float_compat(mpack_token_t t) +{ + mpack_uint32_t sign; + mpack_sint32_t exponent, bias; + unsigned mantbits; + unsigned expbits; + double mant; + + if (t.data.value.lo == 0 && t.data.value.hi == 0) + /* nothing to do */ + return 0; + + if (t.length == 4) mantbits = 23, expbits = 8; + else mantbits = 52, expbits = 11; + bias = (1 << (expbits - 1)) - 1; + + /* restore sign/exponent/mantissa */ + if (mantbits == 52) { + sign = t.data.value.hi >> 31; + exponent = (t.data.value.hi >> 20) & ((1 << 11) - 1); + mant = (t.data.value.hi & ((1 << 20) - 1)) * POW2(32); + mant += t.data.value.lo; + } else { + sign = t.data.value.lo >> 31; + exponent = (t.data.value.lo >> 23) & ((1 << 8) - 1); + mant = t.data.value.lo & ((1 << 23) - 1); + } + + mant /= POW2(mantbits); + if (exponent) mant += 1.0; /* restore leading 1 */ + else exponent = 1; /* subnormal */ + exponent -= bias; + + /* restore original value */ + while (exponent > 0) mant *= 2.0, exponent--; + while (exponent < 0) mant /= 2.0, exponent++; + return mant * (sign ? -1 : 1); +} + +MPACK_API double mpack_unpack_float_fast(mpack_token_t t) +{ + if (t.length == 4) { + union { + float f; + mpack_uint32_t m; + } conv; + conv.m = t.data.value.lo; + return conv.f; + } else { + union { + double d; + mpack_value_t m; + } conv; + conv.m = t.data.value; + + if (mpack_is_be()) { + MPACK_SWAP_VALUE(conv.m); + } + + return conv.d; + } +} + +MPACK_API double mpack_unpack_number(mpack_token_t t) +{ + double rv; + mpack_uint32_t hi, lo; + if (t.type == MPACK_TOKEN_FLOAT) return mpack_unpack_float(t); + assert(t.type == MPACK_TOKEN_UINT || t.type == MPACK_TOKEN_SINT); + hi = t.data.value.hi; + lo = t.data.value.lo; + if (t.type == MPACK_TOKEN_SINT) { + /* same idea as mpack_unpack_sint, except here we shouldn't rely on + * mpack_uintmax_t having 64-bits, operating on the 32-bit words separately. + */ + if (!hi) { + assert(t.length <= 4); + hi = 0; + lo = (~lo & (((mpack_uint32_t)1 << ((t.length * 8) - 1)) - 1)); + } else { + hi = ~hi; + lo = ~lo; + } + lo++; + if (!lo) hi++; + } + rv = (double)lo + POW2(32) * hi; + return t.type == MPACK_TOKEN_SINT ? -rv : rv; +} + +static int mpack_fits_single(double v) +{ + return (float)v == v; +} + +static mpack_value_t mpack_pack_ieee754(double v, unsigned mantbits, + unsigned expbits) +{ + mpack_value_t rv = {0, 0}; + mpack_sint32_t exponent, bias = (1 << (expbits - 1)) - 1; + mpack_uint32_t sign; + double mant; + + if (v == 0) { + rv.lo = 0; + rv.hi = 0; + goto end; + } + + if (v < 0) sign = 1, mant = -v; + else sign = 0, mant = v; + + exponent = 0; + while (mant >= 2.0) mant /= 2.0, exponent++; + while (mant < 1.0 && exponent > -(bias - 1)) mant *= 2.0, exponent--; + + if (mant < 1.0) exponent = -bias; /* subnormal value */ + else mant = mant - 1.0; /* remove leading 1 */ + exponent += bias; + mant *= POW2(mantbits); + + if (mantbits == 52) { + rv.hi = (mpack_uint32_t)(mant / POW2(32)); + rv.lo = (mpack_uint32_t)(mant - rv.hi * POW2(32)); + rv.hi |= ((mpack_uint32_t)exponent << 20) | (sign << 31); + } else if (mantbits == 23) { + rv.hi = 0; + rv.lo = (mpack_uint32_t)mant; + rv.lo |= ((mpack_uint32_t)exponent << 23) | (sign << 31); + } + +end: + return rv; +} + +static int mpack_is_be(void) +{ + union { + mpack_uint32_t i; + char c[sizeof(mpack_uint32_t)]; + } test; + + test.i = 1; + return test.c[0] == 0; +} + +/* this simplified version of `fmod` that returns the remainder of double + * division by 0xffffffff, which is enough for our purposes */ +static double mpack_fmod_pow2_32(double a) +{ + return a - ((double)(mpack_uint32_t)(a / POW2(32)) * POW2(32)); +} diff --git a/src/mpack/conv.h b/src/mpack/conv.h new file mode 100644 index 0000000000..71f14a067e --- /dev/null +++ b/src/mpack/conv.h @@ -0,0 +1,55 @@ +#ifndef MPACK_CONV_H +#define MPACK_CONV_H + +#include "mpack_core.h" + +#if ULLONG_MAX == 0xffffffffffffffff +typedef long long mpack_sintmax_t; +typedef unsigned long long mpack_uintmax_t; +#elif UINT64_MAX == 0xffffffffffffffff +typedef int64_t mpack_sintmax_t; +typedef uint64_t mpack_uintmax_t; +#else +typedef mpack_sint32_t mpack_sintmax_t; +typedef mpack_uint32_t mpack_uintmax_t; +#endif + +#ifndef bool +# define bool unsigned +#endif + +MPACK_API mpack_token_t mpack_pack_nil(void) FUNUSED FPURE; +MPACK_API mpack_token_t mpack_pack_boolean(unsigned v) FUNUSED FPURE; +MPACK_API mpack_token_t mpack_pack_uint(mpack_uintmax_t v) FUNUSED FPURE; +MPACK_API mpack_token_t mpack_pack_sint(mpack_sintmax_t v) FUNUSED FPURE; +MPACK_API mpack_token_t mpack_pack_float_compat(double v) FUNUSED FPURE; +MPACK_API mpack_token_t mpack_pack_float_fast(double v) FUNUSED FPURE; +MPACK_API mpack_token_t mpack_pack_number(double v) FUNUSED FPURE; +MPACK_API mpack_token_t mpack_pack_chunk(const char *p, mpack_uint32_t l) + FUNUSED FPURE FNONULL; +MPACK_API mpack_token_t mpack_pack_str(mpack_uint32_t l) FUNUSED FPURE; +MPACK_API mpack_token_t mpack_pack_bin(mpack_uint32_t l) FUNUSED FPURE; +MPACK_API mpack_token_t mpack_pack_ext(int type, mpack_uint32_t l) + FUNUSED FPURE; +MPACK_API mpack_token_t mpack_pack_array(mpack_uint32_t l) FUNUSED FPURE; +MPACK_API mpack_token_t mpack_pack_map(mpack_uint32_t l) FUNUSED FPURE; +MPACK_API bool mpack_unpack_boolean(mpack_token_t t) FUNUSED FPURE; +MPACK_API mpack_uintmax_t mpack_unpack_uint(mpack_token_t t) FUNUSED FPURE; +MPACK_API mpack_sintmax_t mpack_unpack_sint(mpack_token_t t) FUNUSED FPURE; +MPACK_API double mpack_unpack_float_fast(mpack_token_t t) FUNUSED FPURE; +MPACK_API double mpack_unpack_float_compat(mpack_token_t t) FUNUSED FPURE; +MPACK_API double mpack_unpack_number(mpack_token_t t) FUNUSED FPURE; + +/* The mpack_{pack,unpack}_float_fast functions should work in 99% of the + * platforms. When compiling for a platform where floats don't use ieee754 as + * the internal format, pass + * -Dmpack_{pack,unpack}_float=mpack_{pack,unpack}_float_compat to the + * compiler.*/ +#ifndef mpack_pack_float +# define mpack_pack_float mpack_pack_float_fast +#endif +#ifndef mpack_unpack_float +# define mpack_unpack_float mpack_unpack_float_fast +#endif + +#endif /* MPACK_CONV_H */ diff --git a/src/mpack/lmpack.c b/src/mpack/lmpack.c new file mode 100644 index 0000000000..99207246c8 --- /dev/null +++ b/src/mpack/lmpack.c @@ -0,0 +1,1215 @@ +/* + * This module exports three classes, and each instance of those classes has its + * own private registry for temporary reference storage(keeping state between + * calls). A private registry makes managing memory much easier since all we + * have to do is call luaL_unref passing the registry reference when the + * instance is collected by the __gc metamethod. + * + * This private registry is manipulated with `lmpack_ref` / `lmpack_unref` / + * `lmpack_geti`, which are analogous to `luaL_ref` / `luaL_unref` / + * `lua_rawgeti` but operate on the private registry passed as argument. + * + * In order to simplify debug registry leaks during normal operation(with the + * leak_test.lua script), these `lmpack_*` registry functions will target the + * normal lua registry when MPACK_DEBUG_REGISTRY_LEAK is defined during + * compilation. + */ +#define LUA_LIB +#include <limits.h> +#include <stdlib.h> +#include <string.h> +#include <stdio.h> + +#include <lauxlib.h> +#include <lua.h> +#include <luaconf.h> + +#include "nvim/macros.h" + +#include "lmpack.h" + +#include "rpc.h" + +#define UNPACKER_META_NAME "mpack.Unpacker" +#define PACKER_META_NAME "mpack.Packer" +#define SESSION_META_NAME "mpack.Session" +#define NIL_NAME "mpack.NIL" +#define EMPTY_DICT_NAME "mpack.empty_dict" + +/* + * TODO(tarruda): When targeting lua 5.3 and being compiled with `long long` + * support(not -ansi), we should make use of lua 64 bit integers for + * representing msgpack integers, since `double` can't represent the full range. + */ + +#ifndef luaL_reg +/* Taken from Lua5.1's lauxlib.h */ +#define luaL_reg luaL_Reg +#endif + +#if LUA_VERSION_NUM > 501 +#ifndef luaL_register +#define luaL_register(L,n,f) luaL_setfuncs(L,f,0) +#endif +#endif + +typedef struct { + lua_State *L; + mpack_parser_t *parser; + int reg, ext, unpacking, mtdict; + char *string_buffer; +} Unpacker; + +typedef struct { + lua_State *L; + mpack_parser_t *parser; + int reg, ext, root, packing, mtdict; + int is_bin, is_bin_fn; +} Packer; + +typedef struct { + lua_State *L; + int reg; + mpack_rpc_session_t *session; + struct { + int type; + mpack_rpc_message_t msg; + int method_or_error; + int args_or_result; + } unpacked; + int unpacker; +} Session; + +static int lmpack_ref(lua_State *L, int reg) +{ +#ifdef MPACK_DEBUG_REGISTRY_LEAK + return luaL_ref(L, LUA_REGISTRYINDEX); +#else + int rv; + lua_rawgeti(L, LUA_REGISTRYINDEX, reg); + lua_pushvalue(L, -2); + rv = luaL_ref(L, -2); + lua_pop(L, 2); + return rv; +#endif +} + +static void lmpack_unref(lua_State *L, int reg, int ref) +{ +#ifdef MPACK_DEBUG_REGISTRY_LEAK + luaL_unref(L, LUA_REGISTRYINDEX, ref); +#else + lua_rawgeti(L, LUA_REGISTRYINDEX, reg); + luaL_unref(L, -1, ref); + lua_pop(L, 1); +#endif +} + +static void lmpack_geti(lua_State *L, int reg, int ref) +{ +#ifdef MPACK_DEBUG_REGISTRY_LEAK + lua_rawgeti(L, LUA_REGISTRYINDEX, ref); +#else + lua_rawgeti(L, LUA_REGISTRYINDEX, reg); + lua_rawgeti(L, -1, ref); + lua_replace(L, -2); +#endif +} + +/* make a shallow copy of the table on stack and remove it after the copy is + * done */ +static void lmpack_shallow_copy(lua_State *L) +{ + lua_newtable(L); + lua_pushnil(L); + while (lua_next(L, -3)) { + lua_pushvalue(L, -2); + lua_insert(L, -2); + lua_settable(L, -4); + } + lua_remove(L, -2); +} + +static mpack_parser_t *lmpack_grow_parser(mpack_parser_t *parser) +{ + mpack_parser_t *old = parser; + mpack_uint32_t new_capacity = old->capacity * 2; + parser = malloc(MPACK_PARSER_STRUCT_SIZE(new_capacity)); + if (!parser) goto end; + mpack_parser_init(parser, new_capacity); + mpack_parser_copy(parser, old); + free(old); +end: + return parser; +} + +static mpack_rpc_session_t *lmpack_grow_session(mpack_rpc_session_t *session) +{ + mpack_rpc_session_t *old = session; + mpack_uint32_t new_capacity = old->capacity * 2; + session = malloc(MPACK_RPC_SESSION_STRUCT_SIZE(new_capacity)); + if (!session) goto end; + mpack_rpc_session_init(session, new_capacity); + mpack_rpc_session_copy(session, old); + free(old); +end: + return session; +} + +static Unpacker *lmpack_check_unpacker(lua_State *L, int index) +{ + return luaL_checkudata(L, index, UNPACKER_META_NAME); +} + +static Packer *lmpack_check_packer(lua_State *L, int index) +{ + return luaL_checkudata(L, index, PACKER_META_NAME); +} + +static Session *lmpack_check_session(lua_State *L, int index) +{ + return luaL_checkudata(L, index, SESSION_META_NAME); +} + +static int lmpack_isnil(lua_State *L, int index) +{ + int rv; + if (!lua_isuserdata(L, index)) return 0; + lua_getfield(L, LUA_REGISTRYINDEX, NIL_NAME); + rv = lua_rawequal(L, -1, -2); + lua_pop(L, 1); + return rv; +} + +static int lmpack_isunpacker(lua_State *L, int index) +{ + int rv; + if (!lua_isuserdata(L, index) || !lua_getmetatable(L, index)) return 0; + luaL_getmetatable(L, UNPACKER_META_NAME); + rv = lua_rawequal(L, -1, -2); + lua_pop(L, 2); + return rv; +} + +static void lmpack_pushnil(lua_State *L) +{ + lua_getfield(L, LUA_REGISTRYINDEX, NIL_NAME); +} + +/* adapted from + * https://github.com/antirez/lua-cmsgpack/blob/master/lua_cmsgpack.c */ +static mpack_uint32_t lmpack_objlen(lua_State *L, int *is_array) +{ + size_t len, max; + int isarr, type; + lua_Number n; +#ifndef NDEBUG + int top = lua_gettop(L); + assert(top); +#endif + + if ((type = lua_type(L, -1)) != LUA_TTABLE) { +#if LUA_VERSION_NUM >= 502 + len = lua_rawlen(L, -1); +#elif LUA_VERSION_NUM == 501 + len = lua_objlen(L, -1); +#else + #error You have either broken or too old Lua installation. This library requires Lua>=5.1 +#endif + goto end; + } + + /* count the number of keys and determine if it is an array */ + len = 0; + max = 0; + isarr = 1; + lua_pushnil(L); + + while (lua_next(L, -2)) { + lua_pop(L, 1); /* pop value */ + isarr = isarr + && lua_isnumber(L, -1) /* lua number */ + && (n = lua_tonumber(L, -1)) > 0 /* greater than 0 */ + && (size_t)n == n; /* and integer */ + max = isarr && (size_t)n > max ? (size_t)n : max; + len++; + } + + // when len==0, the caller should guess the type! + if (len > 0) { + *is_array = isarr && max == len; + } + +end: + if ((size_t)-1 > (mpack_uint32_t)-1 && len > (mpack_uint32_t)-1) + /* msgpack spec doesn't allow lengths > 32 bits */ + len = (mpack_uint32_t)-1; + assert(top == lua_gettop(L)); + return (mpack_uint32_t)len; +} + +static int lmpack_unpacker_new(lua_State *L) +{ + Unpacker *rv; + + if (lua_gettop(L) > 1) + return luaL_error(L, "expecting at most 1 table argument"); + + rv = lua_newuserdata(L, sizeof(*rv)); + rv->parser = malloc(sizeof(*rv->parser)); + if (!rv->parser) return luaL_error(L, "Failed to allocate memory"); + mpack_parser_init(rv->parser, 0); + rv->parser->data.p = rv; + rv->string_buffer = NULL; + rv->L = L; + rv->unpacking = 0; + luaL_getmetatable(L, UNPACKER_META_NAME); + lua_setmetatable(L, -2); + +#ifndef MPACK_DEBUG_REGISTRY_LEAK + lua_newtable(L); + rv->reg = luaL_ref(L, LUA_REGISTRYINDEX); +#endif + rv->ext = LUA_NOREF; + + lua_getfield(L, LUA_REGISTRYINDEX, EMPTY_DICT_NAME); + rv->mtdict = lmpack_ref(L, rv->reg); + + if (lua_istable(L, 1)) { + /* parse options */ + lua_getfield(L, 1, "ext"); + if (!lua_isnil(L, -1)) { + if (!lua_istable(L, -1)) + return luaL_error(L, "\"ext\" option must be a table"); + lmpack_shallow_copy(L); + } + rv->ext = lmpack_ref(L, rv->reg); + } + + return 1; +} + +static int lmpack_unpacker_delete(lua_State *L) +{ + Unpacker *unpacker = lmpack_check_unpacker(L, 1); + if (unpacker->ext != LUA_NOREF) + lmpack_unref(L, unpacker->reg, unpacker->ext); +#ifndef MPACK_DEBUG_REGISTRY_LEAK + luaL_unref(L, LUA_REGISTRYINDEX, unpacker->reg); +#endif + free(unpacker->parser); + return 0; +} + +static void lmpack_parse_enter(mpack_parser_t *parser, mpack_node_t *node) +{ + Unpacker *unpacker = parser->data.p; + lua_State *L = unpacker->L; + + switch (node->tok.type) { + case MPACK_TOKEN_NIL: + lmpack_pushnil(L); break; + case MPACK_TOKEN_BOOLEAN: + lua_pushboolean(L, (int)mpack_unpack_boolean(node->tok)); break; + case MPACK_TOKEN_UINT: + case MPACK_TOKEN_SINT: + case MPACK_TOKEN_FLOAT: + lua_pushnumber(L, mpack_unpack_number(node->tok)); break; + case MPACK_TOKEN_CHUNK: + assert(unpacker->string_buffer); + memcpy(unpacker->string_buffer + MPACK_PARENT_NODE(node)->pos, + node->tok.data.chunk_ptr, node->tok.length); + break; + case MPACK_TOKEN_BIN: + case MPACK_TOKEN_STR: + case MPACK_TOKEN_EXT: + unpacker->string_buffer = malloc(node->tok.length); + if (!unpacker->string_buffer) luaL_error(L, "Failed to allocate memory"); + break; + case MPACK_TOKEN_ARRAY: + case MPACK_TOKEN_MAP: + lua_newtable(L); + node->data[0].i = lmpack_ref(L, unpacker->reg); + break; + } +} + +static void lmpack_parse_exit(mpack_parser_t *parser, mpack_node_t *node) +{ + Unpacker *unpacker = parser->data.p; + lua_State *L = unpacker->L; + mpack_node_t *parent = MPACK_PARENT_NODE(node); + + switch (node->tok.type) { + case MPACK_TOKEN_BIN: + case MPACK_TOKEN_STR: + case MPACK_TOKEN_EXT: + lua_pushlstring(L, unpacker->string_buffer, node->tok.length); + free(unpacker->string_buffer); + unpacker->string_buffer = NULL; + if (node->tok.type == MPACK_TOKEN_EXT && unpacker->ext != LUA_NOREF) { + /* check if there's a handler for this type */ + lmpack_geti(L, unpacker->reg, unpacker->ext); + lua_rawgeti(L, -1, node->tok.data.ext_type); + if (lua_isfunction(L, -1)) { + /* stack: + * + * -1: ext unpacker function + * -2: ext unpackers table + * -3: ext string + * + * We want to call the ext unpacker function with the type and string + * as arguments, so push those now + */ + lua_pushinteger(L, node->tok.data.ext_type); + lua_pushvalue(L, -4); + lua_call(L, 2, 1); + /* stack: + * + * -1: returned object + * -2: ext unpackers table + * -3: ext string + */ + lua_replace(L, -3); + } else { + /* the last lua_rawgeti should have pushed nil on the stack, + * remove it */ + lua_pop(L, 1); + } + /* pop the ext unpackers table */ + lua_pop(L, 1); + } + break; + case MPACK_TOKEN_ARRAY: + case MPACK_TOKEN_MAP: + lmpack_geti(L, unpacker->reg, (int)node->data[0].i); + lmpack_unref(L, unpacker->reg, (int)node->data[0].i); + if (node->key_visited == 0 && node->tok.type == MPACK_TOKEN_MAP) { + lmpack_geti(L, unpacker->reg, unpacker->mtdict); // [table, mtdict] + lua_setmetatable(L, -2); // [table] + } + + break; + default: + break; + } + + if (parent && parent->tok.type < MPACK_TOKEN_BIN) { + /* At this point the parsed object is on the stack. Add it to the parent + * container. First put the container on the stack. */ + lmpack_geti(L, unpacker->reg, (int)parent->data[0].i); + + if (parent->tok.type == MPACK_TOKEN_ARRAY) { + /* Array, save the value on key equal to `parent->pos` */ + lua_pushnumber(L, (lua_Number)parent->pos); + lua_pushvalue(L, -3); + lua_settable(L, -3); + } else { + assert(parent->tok.type == MPACK_TOKEN_MAP); + if (parent->key_visited) { + /* save the key on the registry */ + lua_pushvalue(L, -2); + parent->data[1].i = lmpack_ref(L, unpacker->reg); + } else { + /* set the key/value pair */ + lmpack_geti(L, unpacker->reg, (int)parent->data[1].i); + lmpack_unref(L, unpacker->reg, (int)parent->data[1].i); + lua_pushvalue(L, -3); + lua_settable(L, -3); + } + } + lua_pop(L, 2); /* pop the container/object */ + } +} + +static int lmpack_unpacker_unpack_str(lua_State *L, Unpacker *unpacker, + const char **str, size_t *len) +{ + int rv; + + if (unpacker->unpacking) { + return luaL_error(L, "Unpacker instance already working. Use another " + "Unpacker or the module's \"unpack\" function if you " + "need to unpack from the ext handler"); + } + + do { + unpacker->unpacking = 1; + rv = mpack_parse(unpacker->parser, str, len, lmpack_parse_enter, + lmpack_parse_exit); + unpacker->unpacking = 0; + + if (rv == MPACK_NOMEM) { + unpacker->parser = lmpack_grow_parser(unpacker->parser); + if (!unpacker->parser) { + unpacker->unpacking = 0; + return luaL_error(L, "failed to grow Unpacker capacity"); + } + } + } while (rv == MPACK_NOMEM); + + if (rv == MPACK_ERROR) + return luaL_error(L, "invalid msgpack string"); + + return rv; +} + +static int lmpack_unpacker_unpack(lua_State *L) +{ + int result, argc; + lua_Number startpos; + size_t len, offset; + const char *str, *str_init; + Unpacker *unpacker; + + if ((argc = lua_gettop(L)) > 3 || argc < 2) + return luaL_error(L, "expecting between 2 and 3 arguments"); + + unpacker = lmpack_check_unpacker(L, 1); + unpacker->L = L; + + str_init = str = luaL_checklstring(L, 2, &len); + startpos = lua_gettop(L) == 3 ? luaL_checknumber(L, 3) : 1; + + luaL_argcheck(L, startpos > 0, 3, + "start position must be greater than zero"); + luaL_argcheck(L, (size_t)startpos == startpos, 3, + "start position must be an integer"); + luaL_argcheck(L, (size_t)startpos <= len, 3, + "start position must be less than or equal to the input string length"); + + offset = (size_t)startpos - 1 ; + str += offset; + len -= offset; + result = lmpack_unpacker_unpack_str(L, unpacker, &str, &len); + + if (result == MPACK_EOF) + /* if we hit EOF, return nil as the object */ + lua_pushnil(L); + + /* also return the new position in the input string */ + lua_pushinteger(L, str - str_init + 1); + assert(lua_gettop(L) == argc + 2); + return 2; +} + +static int lmpack_packer_new(lua_State *L) +{ + Packer *rv; + + if (lua_gettop(L) > 1) + return luaL_error(L, "expecting at most 1 table argument"); + + rv = lua_newuserdata(L, sizeof(*rv)); + rv->parser = malloc(sizeof(*rv->parser)); + if (!rv->parser) return luaL_error(L, "failed to allocate parser memory"); + mpack_parser_init(rv->parser, 0); + rv->parser->data.p = rv; + rv->L = L; + rv->packing = 0; + rv->is_bin = 0; + rv->is_bin_fn = LUA_NOREF; + luaL_getmetatable(L, PACKER_META_NAME); + lua_setmetatable(L, -2); + +#ifndef MPACK_DEBUG_REGISTRY_LEAK + lua_newtable(L); + rv->reg = luaL_ref(L, LUA_REGISTRYINDEX); +#endif + rv->ext = LUA_NOREF; + + lua_getfield(L, LUA_REGISTRYINDEX, EMPTY_DICT_NAME); + rv->mtdict = lmpack_ref(L, rv->reg); + + if (lua_istable(L, 1)) { + /* parse options */ + lua_getfield(L, 1, "ext"); + if (!lua_isnil(L, -1)) { + if (!lua_istable(L, -1)) + return luaL_error(L, "\"ext\" option must be a table"); + lmpack_shallow_copy(L); + } + rv->ext = lmpack_ref(L, rv->reg); + lua_getfield(L, 1, "is_bin"); + if (!lua_isnil(L, -1)) { + if (!lua_isboolean(L, -1) && !lua_isfunction(L, -1)) + return luaL_error(L, + "\"is_bin\" option must be a boolean or function"); + rv->is_bin = lua_toboolean(L, -1); + if (lua_isfunction(L, -1)) rv->is_bin_fn = lmpack_ref(L, rv->reg); + else lua_pop(L, 1); + } else { + lua_pop(L, 1); + } + + } + + return 1; +} + +static int lmpack_packer_delete(lua_State *L) +{ + Packer *packer = lmpack_check_packer(L, 1); + if (packer->ext != LUA_NOREF) + lmpack_unref(L, packer->reg, packer->ext); +#ifndef MPACK_DEBUG_REGISTRY_LEAK + luaL_unref(L, LUA_REGISTRYINDEX, packer->reg); +#endif + free(packer->parser); + return 0; +} + +static void lmpack_unparse_enter(mpack_parser_t *parser, mpack_node_t *node) +{ + int type; + Packer *packer = parser->data.p; + lua_State *L = packer->L; + mpack_node_t *parent = MPACK_PARENT_NODE(node); + + if (parent) { + /* get the parent */ + lmpack_geti(L, packer->reg, (int)parent->data[0].i); + + if (parent->tok.type > MPACK_TOKEN_MAP) { + /* strings are a special case, they are packed as single child chunk + * node */ + const char *str = lua_tolstring(L, -1, NULL); + node->tok = mpack_pack_chunk(str, parent->tok.length); + lua_pop(L, 1); + return; + } + + if (parent->tok.type == MPACK_TOKEN_ARRAY) { + /* push the next index */ + lua_pushnumber(L, (lua_Number)(parent->pos + 1)); + /* push the element */ + lua_gettable(L, -2); + } else if (parent->tok.type == MPACK_TOKEN_MAP) { + int result; + /* push the previous key */ + lmpack_geti(L, packer->reg, (int)parent->data[1].i); + /* push the pair */ + result = lua_next(L, -2); + assert(result); /* should not be here if the map was fully processed */ + if (parent->key_visited) { + /* release the current key */ + lmpack_unref(L, packer->reg, (int)parent->data[1].i); + /* push key to the top */ + lua_pushvalue(L, -2); + /* set the key for the next iteration, leaving value on top */ + parent->data[1].i = lmpack_ref(L, packer->reg); + /* replace key by the value */ + lua_replace(L, -2); + } else { + /* pop value */ + lua_pop(L, 1); + } + } + /* remove parent, leaving only the object which will be serialized */ + lua_remove(L, -2); + } else { + /* root object */ + lmpack_geti(L, packer->reg, packer->root); + } + + type = lua_type(L, -1); + + switch (type) { + case LUA_TBOOLEAN: + node->tok = mpack_pack_boolean((unsigned)lua_toboolean(L, -1)); + break; + case LUA_TNUMBER: + node->tok = mpack_pack_number(lua_tonumber(L, -1)); + break; + case LUA_TSTRING: { + int is_bin = packer->is_bin; + if (is_bin && packer->is_bin_fn != LUA_NOREF) { + lmpack_geti(L, packer->reg, packer->is_bin_fn); + lua_pushvalue(L, -2); + lua_call(L, 1, 1); + is_bin = lua_toboolean(L, -1); + lua_pop(L, 1); + } + if (is_bin) node->tok = mpack_pack_bin(lmpack_objlen(L, NULL)); + else node->tok = mpack_pack_str(lmpack_objlen(L, NULL)); + break; + } + case LUA_TTABLE: { + mpack_uint32_t len; + mpack_node_t *n; + + int has_meta = lua_getmetatable(L, -1); + if (packer->ext != LUA_NOREF && has_meta) { + /* check if there's a handler for this metatable */ + lmpack_geti(L, packer->reg, packer->ext); + lua_pushvalue(L, -2); + lua_gettable(L, -2); + if (lua_isfunction(L, -1)) { + lua_Number ext = -1; + /* stack: + * + * -1: ext packer function + * -2: ext packers table + * -3: metatable + * -4: original object + * + * We want to call the ext packer function with the original object as + * argument, so push it on the top + */ + lua_pushvalue(L, -4); + /* handler should return type code and string */ + lua_call(L, 1, 2); + if (!lua_isnumber(L, -2) || (ext = lua_tonumber(L, -2)) < 0 + || ext > 127 || (int)ext != ext) + luaL_error(L, + "the first result from ext packer must be an integer " + "between 0 and 127"); + if (!lua_isstring(L, -1)) + luaL_error(L, + "the second result from ext packer must be a string"); + node->tok = mpack_pack_ext((int)ext, lmpack_objlen(L, NULL)); + /* stack: + * + * -1: ext string + * -2: ext type + * -3: ext packers table + * -4: metatable + * -5: original table + * + * We want to leave only the returned ext string, so + * replace -5 with the string and pop 3 + */ + lua_replace(L, -5); + lua_pop(L, 3); + break; /* done */ + } else { + /* stack: + * + * -1: ext packers table + * -2: metatable + * -3: original table + * + * We want to leave only the original table and metatable since they + * will be handled below, so pop 1 + */ + lua_pop(L, 1); + } + } + + int is_array = 1; + if (has_meta) { + // stack: [table, metatable] + if (packer->mtdict != LUA_NOREF) { + lmpack_geti(L, packer->reg, packer->mtdict); // [table, metatable, mtdict] + is_array = !lua_rawequal(L, -1, -2); + lua_pop(L, 1); // [table, metatable]; + } + lua_pop(L, 1); // [table] + } + + /* check for cycles */ + n = node; + while ((n = MPACK_PARENT_NODE(n))) { + lmpack_geti(L, packer->reg, (int)n->data[0].i); + if (lua_rawequal(L, -1, -2)) { + /* break out of cycles with NIL */ + node->tok = mpack_pack_nil(); + lua_pop(L, 2); + lmpack_pushnil(L); + goto end; + } + lua_pop(L, 1); + } + + len = lmpack_objlen(L, &is_array); + if (is_array) { + node->tok = mpack_pack_array(len); + } else { + node->tok = mpack_pack_map(len); + /* save nil as the previous key to start iteration */ + node->data[1].i = LUA_REFNIL; + } + break; + } + case LUA_TUSERDATA: + if (lmpack_isnil(L, -1)) { + node->tok = mpack_pack_nil(); + break; + } + FALLTHROUGH; + default: + { + /* #define FMT */ + char errmsg[50]; + snprintf(errmsg, 50, "can't serialize object of type %d", type); + luaL_error(L, errmsg); + } + } + +end: + node->data[0].i = lmpack_ref(L, packer->reg); +} + +static void lmpack_unparse_exit(mpack_parser_t *parser, mpack_node_t *node) +{ + Packer *packer = parser->data.p; + lua_State *L = packer->L; + if (node->tok.type != MPACK_TOKEN_CHUNK) { + /* release the object */ + lmpack_unref(L, packer->reg, (int)node->data[0].i); + if (node->tok.type == MPACK_TOKEN_MAP) + lmpack_unref(L, packer->reg, (int)node->data[1].i); + } +} + +static int lmpack_packer_pack(lua_State *L) +{ + char *b; + size_t bl; + int result, argc; + Packer *packer; + luaL_Buffer buffer; + + if ((argc = lua_gettop(L)) != 2) + return luaL_error(L, "expecting exactly 2 arguments"); + + packer = lmpack_check_packer(L, 1); + packer->L = L; + packer->root = lmpack_ref(L, packer->reg); + luaL_buffinit(L, &buffer); + b = luaL_prepbuffer(&buffer); + bl = LUAL_BUFFERSIZE; + + if (packer->packing) { + return luaL_error(L, "Packer instance already working. Use another Packer " + "or the module's \"pack\" function if you need to " + "pack from the ext handler"); + } + + do { + size_t bl_init = bl; + packer->packing = 1; + result = mpack_unparse(packer->parser, &b, &bl, lmpack_unparse_enter, + lmpack_unparse_exit); + packer->packing = 0; + + if (result == MPACK_NOMEM) { + packer->parser = lmpack_grow_parser(packer->parser); + if (!packer->parser) { + packer->packing = 0; + return luaL_error(L, "Failed to grow Packer capacity"); + } + } + + luaL_addsize(&buffer, bl_init - bl); + + if (!bl) { + /* buffer empty, resize */ + b = luaL_prepbuffer(&buffer); + bl = LUAL_BUFFERSIZE; + } + } while (result == MPACK_EOF || result == MPACK_NOMEM); + + lmpack_unref(L, packer->reg, packer->root); + luaL_pushresult(&buffer); + assert(lua_gettop(L) == argc); + return 1; +} + +static int lmpack_session_new(lua_State *L) +{ + Session *rv = lua_newuserdata(L, sizeof(*rv)); + rv->session = malloc(sizeof(*rv->session)); + if (!rv->session) return luaL_error(L, "Failed to allocate memory"); + mpack_rpc_session_init(rv->session, 0); + rv->L = L; + luaL_getmetatable(L, SESSION_META_NAME); + lua_setmetatable(L, -2); +#ifndef MPACK_DEBUG_REGISTRY_LEAK + lua_newtable(L); + rv->reg = luaL_ref(L, LUA_REGISTRYINDEX); +#endif + rv->unpacker = LUA_REFNIL; + rv->unpacked.args_or_result = LUA_NOREF; + rv->unpacked.method_or_error = LUA_NOREF; + rv->unpacked.type = MPACK_EOF; + + if (lua_istable(L, 1)) { + /* parse options */ + lua_getfield(L, 1, "unpack"); + if (!lmpack_isunpacker(L, -1)) { + return luaL_error(L, + "\"unpack\" option must be a " UNPACKER_META_NAME " instance"); + } + rv->unpacker = lmpack_ref(L, rv->reg); + } + + return 1; +} + +static int lmpack_session_delete(lua_State *L) +{ + Session *session = lmpack_check_session(L, 1); + lmpack_unref(L, session->reg, session->unpacker); +#ifndef MPACK_DEBUG_REGISTRY_LEAK + luaL_unref(L, LUA_REGISTRYINDEX, session->reg); +#endif + free(session->session); + return 0; +} + +static int lmpack_session_receive(lua_State *L) +{ + int argc, done, rcount = 3; + lua_Number startpos; + size_t len; + const char *str, *str_init; + Session *session; + Unpacker *unpacker = NULL; + + if ((argc = lua_gettop(L)) > 3 || argc < 2) + return luaL_error(L, "expecting between 2 and 3 arguments"); + + session = lmpack_check_session(L, 1); + str_init = str = luaL_checklstring(L, 2, &len); + startpos = lua_gettop(L) == 3 ? luaL_checknumber(L, 3) : 1; + + luaL_argcheck(L, startpos > 0, 3, + "start position must be greater than zero"); + luaL_argcheck(L, (size_t)startpos == startpos, 3, + "start position must be an integer"); + luaL_argcheck(L, (size_t)startpos <= len, 3, + "start position must be less than or equal to the input string length"); + + str += (size_t)startpos - 1; + + if (session->unpacker != LUA_REFNIL) { + lmpack_geti(L, session->reg, session->unpacker); + unpacker = lmpack_check_unpacker(L, -1); + unpacker->L = L; + rcount += 2; + lua_pop(L, 1); + } + + for (;;) { + int result; + + if (session->unpacked.type == MPACK_EOF) { + session->unpacked.type = + mpack_rpc_receive(session->session, &str, &len, &session->unpacked.msg); + + if (!unpacker || session->unpacked.type == MPACK_EOF) + break; + } + + result = lmpack_unpacker_unpack_str(L, unpacker, &str, &len); + + if (result == MPACK_EOF) break; + + if (session->unpacked.method_or_error == LUA_NOREF) { + session->unpacked.method_or_error = lmpack_ref(L, session->reg); + } else { + session->unpacked.args_or_result = lmpack_ref(L, session->reg); + break; + } + } + + done = session->unpacked.type != MPACK_EOF + && (session->unpacked.args_or_result != LUA_NOREF || !unpacker); + + if (!done) { + lua_pushnil(L); + lua_pushnil(L); + if (unpacker) { + lua_pushnil(L); + lua_pushnil(L); + } + goto end; + } + + switch (session->unpacked.type) { + case MPACK_RPC_REQUEST: + lua_pushstring(L, "request"); + lua_pushnumber(L, session->unpacked.msg.id); + break; + case MPACK_RPC_RESPONSE: + lua_pushstring(L, "response"); + lmpack_geti(L, session->reg, (int)session->unpacked.msg.data.i); + break; + case MPACK_RPC_NOTIFICATION: + lua_pushstring(L, "notification"); + lua_pushnil(L); + break; + default: + /* In most cases the only sane thing to do when receiving invalid + * msgpack-rpc is to close the connection, so handle all errors with + * this generic message. Later may add more detailed information. */ + return luaL_error(L, "invalid msgpack-rpc string"); + } + + session->unpacked.type = MPACK_EOF; + + if (unpacker) { + lmpack_geti(L, session->reg, session->unpacked.method_or_error); + lmpack_geti(L, session->reg, session->unpacked.args_or_result); + lmpack_unref(L, session->reg, session->unpacked.method_or_error); + lmpack_unref(L, session->reg, session->unpacked.args_or_result); + session->unpacked.method_or_error = LUA_NOREF; + session->unpacked.args_or_result = LUA_NOREF; + } + +end: + lua_pushinteger(L, str - str_init + 1); + return rcount; +} + +static int lmpack_session_request(lua_State *L) +{ + int result; + char buf[16], *b = buf; + size_t bl = sizeof(buf); + Session *session; + mpack_data_t data; + + if (lua_gettop(L) > 2 || lua_gettop(L) < 1) + return luaL_error(L, "expecting 1 or 2 arguments"); + + session = lmpack_check_session(L, 1); + data.i = lua_isnoneornil(L, 2) ? LUA_NOREF : lmpack_ref(L, session->reg); + do { + result = mpack_rpc_request(session->session, &b, &bl, data); + if (result == MPACK_NOMEM) { + session->session = lmpack_grow_session(session->session); + if (!session->session) + return luaL_error(L, "Failed to grow Session capacity"); + } + } while (result == MPACK_NOMEM); + + assert(result == MPACK_OK); + lua_pushlstring(L, buf, sizeof(buf) - bl); + return 1; +} + +static int lmpack_session_reply(lua_State *L) +{ + int result; + char buf[16], *b = buf; + size_t bl = sizeof(buf); + Session *session; + lua_Number id; + + if (lua_gettop(L) != 2) + return luaL_error(L, "expecting exactly 2 arguments"); + + session = lmpack_check_session(L, 1); + id = lua_tonumber(L, 2); + luaL_argcheck(L, ((size_t)id == id && id >= 0 && id <= 0xffffffff), 2, + "invalid request id"); + result = mpack_rpc_reply(session->session, &b, &bl, (mpack_uint32_t)id); + assert(result == MPACK_OK); + lua_pushlstring(L, buf, sizeof(buf) - bl); + return 1; +} + +static int lmpack_session_notify(lua_State *L) +{ + int result; + char buf[16], *b = buf; + size_t bl = sizeof(buf); + Session *session; + + if (lua_gettop(L) != 1) + return luaL_error(L, "expecting exactly 1 argument"); + + session = lmpack_check_session(L, 1); + result = mpack_rpc_notify(session->session, &b, &bl); + assert(result == MPACK_OK); + lua_pushlstring(L, buf, sizeof(buf) - bl); + return 1; +} + +static int lmpack_nil_tostring(lua_State* L) +{ + lua_pushfstring(L, NIL_NAME, lua_topointer(L, 1)); + return 1; +} + +static int lmpack_unpack(lua_State *L) +{ + int result; + size_t len; + const char *str; + Unpacker unpacker; + mpack_parser_t parser; + + if (lua_gettop(L) != 1) + return luaL_error(L, "expecting exactly 1 argument"); + + str = luaL_checklstring(L, 1, &len); + + /* initialize unpacker */ + lua_newtable(L); + unpacker.reg = luaL_ref(L, LUA_REGISTRYINDEX); + unpacker.ext = LUA_NOREF; + unpacker.parser = &parser; + mpack_parser_init(unpacker.parser, 0); + unpacker.parser->data.p = &unpacker; + unpacker.string_buffer = NULL; + unpacker.L = L; + + lua_getfield(L, LUA_REGISTRYINDEX, EMPTY_DICT_NAME); + unpacker.mtdict = lmpack_ref(L, unpacker.reg); + + result = mpack_parse(&parser, &str, &len, lmpack_parse_enter, + lmpack_parse_exit); + + luaL_unref(L, LUA_REGISTRYINDEX, unpacker.reg); + + if (result == MPACK_NOMEM) + return luaL_error(L, "object was too deep to unpack"); + else if (result == MPACK_EOF) + return luaL_error(L, "incomplete msgpack string"); + else if (result == MPACK_ERROR) + return luaL_error(L, "invalid msgpack string"); + else if (result == MPACK_OK && len) + return luaL_error(L, "trailing data in msgpack string"); + + assert(result == MPACK_OK); + return 1; +} + +static int lmpack_pack(lua_State *L) +{ + char *b; + size_t bl; + int result; + Packer packer; + mpack_parser_t parser; + luaL_Buffer buffer; + + if (lua_gettop(L) != 1) + return luaL_error(L, "expecting exactly 1 argument"); + + /* initialize packer */ + lua_newtable(L); + packer.reg = luaL_ref(L, LUA_REGISTRYINDEX); + packer.ext = LUA_NOREF; + packer.parser = &parser; + mpack_parser_init(packer.parser, 0); + packer.parser->data.p = &packer; + packer.is_bin = 0; + packer.L = L; + packer.root = lmpack_ref(L, packer.reg); + + lua_getfield(L, LUA_REGISTRYINDEX, EMPTY_DICT_NAME); + packer.mtdict = lmpack_ref(L, packer.reg); + + + luaL_buffinit(L, &buffer); + b = luaL_prepbuffer(&buffer); + bl = LUAL_BUFFERSIZE; + + do { + size_t bl_init = bl; + result = mpack_unparse(packer.parser, &b, &bl, lmpack_unparse_enter, + lmpack_unparse_exit); + + if (result == MPACK_NOMEM) { + lmpack_unref(L, packer.reg, packer.root); + luaL_unref(L, LUA_REGISTRYINDEX, packer.reg); + return luaL_error(L, "object was too deep to pack"); + } + + luaL_addsize(&buffer, bl_init - bl); + + if (!bl) { + /* buffer empty, resize */ + b = luaL_prepbuffer(&buffer); + bl = LUAL_BUFFERSIZE; + } + } while (result == MPACK_EOF); + + lmpack_unref(L, packer.reg, packer.root); + luaL_unref(L, LUA_REGISTRYINDEX, packer.reg); + luaL_pushresult(&buffer); + return 1; +} + +static const luaL_reg unpacker_methods[] = { + {"__call", lmpack_unpacker_unpack}, + {"__gc", lmpack_unpacker_delete}, + {NULL, NULL} +}; + +static const luaL_reg packer_methods[] = { + {"__call", lmpack_packer_pack}, + {"__gc", lmpack_packer_delete}, + {NULL, NULL} +}; + +static const luaL_reg session_methods[] = { + {"receive", lmpack_session_receive}, + {"request", lmpack_session_request}, + {"reply", lmpack_session_reply}, + {"notify", lmpack_session_notify}, + {"__gc", lmpack_session_delete}, + {NULL, NULL} +}; + +static const luaL_reg mpack_functions[] = { + {"Unpacker", lmpack_unpacker_new}, + {"Packer", lmpack_packer_new}, + {"Session", lmpack_session_new}, + {"unpack", lmpack_unpack}, + {"pack", lmpack_pack}, + {NULL, NULL} +}; + +int luaopen_mpack(lua_State *L) +{ + /* Unpacker */ + luaL_newmetatable(L, UNPACKER_META_NAME); + lua_pushvalue(L, -1); + lua_setfield(L, -2, "__index"); + luaL_register(L, NULL, unpacker_methods); + lua_pop(L, 1); + /* Packer */ + luaL_newmetatable(L, PACKER_META_NAME); + lua_pushvalue(L, -1); + lua_setfield(L, -2, "__index"); + luaL_register(L, NULL, packer_methods); + lua_pop(L, 1); + /* Session */ + luaL_newmetatable(L, SESSION_META_NAME); + lua_pushvalue(L, -1); + lua_setfield(L, -2, "__index"); + luaL_register(L, NULL, session_methods); + lua_pop(L, 1); + /* NIL */ + /* Check if NIL is already stored in the registry */ + lua_getfield(L, LUA_REGISTRYINDEX, NIL_NAME); + /* If it isn't, create it */ + if (lua_isnil(L, -1)) { + /* Use a constant userdata to represent NIL */ + (void)lua_newuserdata(L, sizeof(void *)); + /* Create a metatable for NIL userdata */ + lua_createtable(L, 0, 1); + lua_pushstring(L, "__tostring"); + lua_pushcfunction(L, lmpack_nil_tostring); + lua_settable(L, -3); + /* Assign the metatable to the userdata object */ + lua_setmetatable(L, -2); + /* Save NIL on the registry so we can access it easily from other functions */ + lua_setfield(L, LUA_REGISTRYINDEX, NIL_NAME); + } + + lua_pop(L, 1); + + /* module */ + lua_newtable(L); + luaL_register(L, NULL, mpack_functions); + /* save NIL on the module */ + lua_getfield(L, LUA_REGISTRYINDEX, NIL_NAME); + lua_setfield(L, -2, "NIL"); + return 1; +} diff --git a/src/mpack/lmpack.h b/src/mpack/lmpack.h new file mode 100644 index 0000000000..e35f40fab6 --- /dev/null +++ b/src/mpack/lmpack.h @@ -0,0 +1,3 @@ +#include <lua.h> + +int luaopen_mpack(lua_State *L); diff --git a/src/mpack/mpack_core.c b/src/mpack/mpack_core.c new file mode 100644 index 0000000000..0ad09bd46a --- /dev/null +++ b/src/mpack/mpack_core.c @@ -0,0 +1,575 @@ +#include <string.h> + +#include "mpack_core.h" + +#define UNUSED(p) (void)p; +#define ADVANCE(buf, buflen) ((*buflen)--, (unsigned char)*((*buf)++)) +#define TLEN(val, range_start) ((mpack_uint32_t)(1 << (val - range_start))) +#ifndef MIN +# define MIN(X, Y) ((X) < (Y) ? (X) : (Y)) +#endif + +static int mpack_rtoken(const char **buf, size_t *buflen, + mpack_token_t *tok); +static int mpack_rpending(const char **b, size_t *nl, mpack_tokbuf_t *tb); +static int mpack_rvalue(mpack_token_type_t t, mpack_uint32_t l, + const char **b, size_t *bl, mpack_token_t *tok); +static int mpack_rblob(mpack_token_type_t t, mpack_uint32_t l, + const char **b, size_t *bl, mpack_token_t *tok); +static int mpack_wtoken(const mpack_token_t *tok, char **b, size_t *bl); +static int mpack_wpending(char **b, size_t *bl, mpack_tokbuf_t *tb); +static int mpack_wpint(char **b, size_t *bl, mpack_value_t v); +static int mpack_wnint(char **b, size_t *bl, mpack_value_t v); +static int mpack_wfloat(char **b, size_t *bl, const mpack_token_t *v); +static int mpack_wstr(char **buf, size_t *buflen, mpack_uint32_t len); +static int mpack_wbin(char **buf, size_t *buflen, mpack_uint32_t len); +static int mpack_wext(char **buf, size_t *buflen, int type, + mpack_uint32_t len); +static int mpack_warray(char **buf, size_t *buflen, mpack_uint32_t len); +static int mpack_wmap(char **buf, size_t *buflen, mpack_uint32_t len); +static int mpack_w1(char **b, size_t *bl, mpack_uint32_t v); +static int mpack_w2(char **b, size_t *bl, mpack_uint32_t v); +static int mpack_w4(char **b, size_t *bl, mpack_uint32_t v); +static mpack_value_t mpack_byte(unsigned char b); +static int mpack_value(mpack_token_type_t t, mpack_uint32_t l, + mpack_value_t v, mpack_token_t *tok); +static int mpack_blob(mpack_token_type_t t, mpack_uint32_t l, int et, + mpack_token_t *tok); + +MPACK_API void mpack_tokbuf_init(mpack_tokbuf_t *tokbuf) +{ + tokbuf->ppos = 0; + tokbuf->plen = 0; + tokbuf->passthrough = 0; +} + +MPACK_API int mpack_read(mpack_tokbuf_t *tokbuf, const char **buf, + size_t *buflen, mpack_token_t *tok) +{ + int status; + size_t initial_ppos, ptrlen, advanced; + const char *ptr, *ptr_save; + assert(*buf && *buflen); + + if (tokbuf->passthrough) { + /* pass data from str/bin/ext directly as a MPACK_TOKEN_CHUNK, adjusting + * *buf and *buflen */ + tok->type = MPACK_TOKEN_CHUNK; + tok->data.chunk_ptr = *buf; + tok->length = MIN((mpack_uint32_t)*buflen, tokbuf->passthrough); + tokbuf->passthrough -= tok->length; + *buf += tok->length; + *buflen -= tok->length; + goto done; + } + + initial_ppos = tokbuf->ppos; + + if (tokbuf->plen) { + if (!mpack_rpending(buf, buflen, tokbuf)) { + return MPACK_EOF; + } + ptr = tokbuf->pending; + ptrlen = tokbuf->ppos; + } else { + ptr = *buf; + ptrlen = *buflen; + } + + ptr_save = ptr; + + if ((status = mpack_rtoken(&ptr, &ptrlen, tok))) { + if (status != MPACK_EOF) return MPACK_ERROR; + /* need more data */ + assert(!tokbuf->plen); + /* read the remainder of *buf to tokbuf->pending so it can be parsed + * later with more data. only required when tokbuf->plen == 0 or else + * it would have been done already. */ + tokbuf->plen = tok->length + 1; + assert(tokbuf->plen <= sizeof(tokbuf->pending)); + tokbuf->ppos = 0; + status = mpack_rpending(buf, buflen, tokbuf); + assert(!status); + return MPACK_EOF; + } + + advanced = (size_t)(ptr - ptr_save) - initial_ppos; + tokbuf->plen = tokbuf->ppos = 0; + *buflen -= advanced; + *buf += advanced; + + if (tok->type > MPACK_TOKEN_MAP) { + tokbuf->passthrough = tok->length; + } + +done: + return MPACK_OK; +} + +MPACK_API int mpack_write(mpack_tokbuf_t *tokbuf, char **buf, size_t *buflen, + const mpack_token_t *t) +{ + int status; + char *ptr; + size_t ptrlen; + mpack_token_t tok = tokbuf->plen ? tokbuf->pending_tok : *t; + assert(*buf && *buflen); + + if (tok.type == MPACK_TOKEN_CHUNK) { + size_t written, pending, count; + if (!tokbuf->plen) tokbuf->ppos = 0; + written = tokbuf->ppos; + pending = tok.length - written; + count = MIN(pending, *buflen); + memcpy(*buf, tok.data.chunk_ptr + written, count); + *buf += count; + *buflen -= count; + tokbuf->ppos += count; + tokbuf->plen = count == pending ? 0 : tok.length; + if (count == pending) { + return MPACK_OK; + } else { + tokbuf->pending_tok = tok; + return MPACK_EOF; + } + } + + if (tokbuf->plen) return mpack_wpending(buf, buflen, tokbuf); + + if (*buflen < MPACK_MAX_TOKEN_LEN) { + ptr = tokbuf->pending; + ptrlen = sizeof(tokbuf->pending); + } else { + ptr = *buf; + ptrlen = *buflen; + } + + if ((status = mpack_wtoken(&tok, &ptr, &ptrlen))) return status; + + if (*buflen < MPACK_MAX_TOKEN_LEN) { + size_t toklen = sizeof(tokbuf->pending) - ptrlen; + size_t write_cnt = MIN(toklen, *buflen); + memcpy(*buf, tokbuf->pending, write_cnt); + *buf += write_cnt; + *buflen -= write_cnt; + if (write_cnt < toklen) { + assert(!*buflen); + tokbuf->plen = toklen; + tokbuf->ppos = write_cnt; + tokbuf->pending_tok = tok; + return MPACK_EOF; + } + } else { + *buflen -= (size_t)(ptr - *buf); + *buf = ptr; + } + + return MPACK_OK; +} + +static int mpack_rtoken(const char **buf, size_t *buflen, + mpack_token_t *tok) +{ + unsigned char t = ADVANCE(buf, buflen); + if (t < 0x80) { + /* positive fixint */ + return mpack_value(MPACK_TOKEN_UINT, 1, mpack_byte(t), tok); + } else if (t < 0x90) { + /* fixmap */ + return mpack_blob(MPACK_TOKEN_MAP, t & 0xf, 0, tok); + } else if (t < 0xa0) { + /* fixarray */ + return mpack_blob(MPACK_TOKEN_ARRAY, t & 0xf, 0, tok); + } else if (t < 0xc0) { + /* fixstr */ + return mpack_blob(MPACK_TOKEN_STR, t & 0x1f, 0, tok); + } else if (t < 0xe0) { + switch (t) { + case 0xc0: /* nil */ + return mpack_value(MPACK_TOKEN_NIL, 0, mpack_byte(0), tok); + case 0xc2: /* false */ + return mpack_value(MPACK_TOKEN_BOOLEAN, 1, mpack_byte(0), tok); + case 0xc3: /* true */ + return mpack_value(MPACK_TOKEN_BOOLEAN, 1, mpack_byte(1), tok); + case 0xc4: /* bin 8 */ + case 0xc5: /* bin 16 */ + case 0xc6: /* bin 32 */ + return mpack_rblob(MPACK_TOKEN_BIN, TLEN(t, 0xc4), buf, buflen, tok); + case 0xc7: /* ext 8 */ + case 0xc8: /* ext 16 */ + case 0xc9: /* ext 32 */ + return mpack_rblob(MPACK_TOKEN_EXT, TLEN(t, 0xc7), buf, buflen, tok); + case 0xca: /* float 32 */ + case 0xcb: /* float 64 */ + return mpack_rvalue(MPACK_TOKEN_FLOAT, TLEN(t, 0xc8), buf, buflen, tok); + case 0xcc: /* uint 8 */ + case 0xcd: /* uint 16 */ + case 0xce: /* uint 32 */ + case 0xcf: /* uint 64 */ + return mpack_rvalue(MPACK_TOKEN_UINT, TLEN(t, 0xcc), buf, buflen, tok); + case 0xd0: /* int 8 */ + case 0xd1: /* int 16 */ + case 0xd2: /* int 32 */ + case 0xd3: /* int 64 */ + return mpack_rvalue(MPACK_TOKEN_SINT, TLEN(t, 0xd0), buf, buflen, tok); + case 0xd4: /* fixext 1 */ + case 0xd5: /* fixext 2 */ + case 0xd6: /* fixext 4 */ + case 0xd7: /* fixext 8 */ + case 0xd8: /* fixext 16 */ + if (*buflen == 0) { + /* require only one extra byte for the type code */ + tok->length = 1; + return MPACK_EOF; + } + tok->length = TLEN(t, 0xd4); + tok->type = MPACK_TOKEN_EXT; + tok->data.ext_type = ADVANCE(buf, buflen); + return MPACK_OK; + case 0xd9: /* str 8 */ + case 0xda: /* str 16 */ + case 0xdb: /* str 32 */ + return mpack_rblob(MPACK_TOKEN_STR, TLEN(t, 0xd9), buf, buflen, tok); + case 0xdc: /* array 16 */ + case 0xdd: /* array 32 */ + return mpack_rblob(MPACK_TOKEN_ARRAY, TLEN(t, 0xdb), buf, buflen, tok); + case 0xde: /* map 16 */ + case 0xdf: /* map 32 */ + return mpack_rblob(MPACK_TOKEN_MAP, TLEN(t, 0xdd), buf, buflen, tok); + default: + return MPACK_ERROR; + } + } else { + /* negative fixint */ + return mpack_value(MPACK_TOKEN_SINT, 1, mpack_byte(t), tok); + } +} + +static int mpack_rpending(const char **buf, size_t *buflen, + mpack_tokbuf_t *state) +{ + size_t count; + assert(state->ppos < state->plen); + count = MIN(state->plen - state->ppos, *buflen); + memcpy(state->pending + state->ppos, *buf, count); + state->ppos += count; + if (state->ppos < state->plen) { + /* consume buffer since no token will be parsed yet. */ + *buf += *buflen; + *buflen = 0; + return 0; + } + return 1; +} + +static int mpack_rvalue(mpack_token_type_t type, mpack_uint32_t remaining, + const char **buf, size_t *buflen, mpack_token_t *tok) +{ + if (*buflen < remaining) { + tok->length = remaining; + return MPACK_EOF; + } + + mpack_value(type, remaining, mpack_byte(0), tok); + + while (remaining) { + mpack_uint32_t byte = ADVANCE(buf, buflen), byte_idx, byte_shift; + byte_idx = (mpack_uint32_t)--remaining; + byte_shift = (byte_idx % 4) * 8; + tok->data.value.lo |= byte << byte_shift; + if (remaining == 4) { + /* unpacked the first half of a 8-byte value, shift what was parsed to the + * "hi" field and reset "lo" for the trailing 4 bytes. */ + tok->data.value.hi = tok->data.value.lo; + tok->data.value.lo = 0; + } + } + + if (type == MPACK_TOKEN_SINT) { + mpack_uint32_t hi = tok->data.value.hi; + mpack_uint32_t lo = tok->data.value.lo; + mpack_uint32_t msb = (tok->length == 8 && hi >> 31) || + (tok->length == 4 && lo >> 31) || + (tok->length == 2 && lo >> 15) || + (tok->length == 1 && lo >> 7); + if (!msb) { + tok->type = MPACK_TOKEN_UINT; + } + } + + return MPACK_OK; +} + +static int mpack_rblob(mpack_token_type_t type, mpack_uint32_t tlen, + const char **buf, size_t *buflen, mpack_token_t *tok) +{ + mpack_token_t l; + mpack_uint32_t required = tlen + (type == MPACK_TOKEN_EXT ? 1 : 0); + + if (*buflen < required) { + tok->length = required; + return MPACK_EOF; + } + + l.data.value.lo = 0; + mpack_rvalue(MPACK_TOKEN_UINT, tlen, buf, buflen, &l); + tok->type = type; + tok->length = l.data.value.lo; + + if (type == MPACK_TOKEN_EXT) { + tok->data.ext_type = ADVANCE(buf, buflen); + } + + return MPACK_OK; +} + +static int mpack_wtoken(const mpack_token_t *tok, char **buf, + size_t *buflen) +{ + switch (tok->type) { + case MPACK_TOKEN_NIL: + return mpack_w1(buf, buflen, 0xc0); + case MPACK_TOKEN_BOOLEAN: + return mpack_w1(buf, buflen, tok->data.value.lo ? 0xc3 : 0xc2); + case MPACK_TOKEN_UINT: + return mpack_wpint(buf, buflen, tok->data.value); + case MPACK_TOKEN_SINT: + return mpack_wnint(buf, buflen, tok->data.value); + case MPACK_TOKEN_FLOAT: + return mpack_wfloat(buf, buflen, tok); + case MPACK_TOKEN_BIN: + return mpack_wbin(buf, buflen, tok->length); + case MPACK_TOKEN_STR: + return mpack_wstr(buf, buflen, tok->length); + case MPACK_TOKEN_EXT: + return mpack_wext(buf, buflen, tok->data.ext_type, tok->length); + case MPACK_TOKEN_ARRAY: + return mpack_warray(buf, buflen, tok->length); + case MPACK_TOKEN_MAP: + return mpack_wmap(buf, buflen, tok->length); + default: + return MPACK_ERROR; + } +} + +static int mpack_wpending(char **buf, size_t *buflen, mpack_tokbuf_t *state) +{ + size_t count; + assert(state->ppos < state->plen); + count = MIN(state->plen - state->ppos, *buflen); + memcpy(*buf, state->pending + state->ppos, count); + state->ppos += count; + *buf += count; + *buflen -= count; + if (state->ppos == state->plen) { + state->plen = 0; + return MPACK_OK; + } + return MPACK_EOF; +} + +static int mpack_wpint(char **buf, size_t *buflen, mpack_value_t val) +{ + mpack_uint32_t hi = val.hi; + mpack_uint32_t lo = val.lo; + + if (hi) { + /* uint 64 */ + return mpack_w1(buf, buflen, 0xcf) || + mpack_w4(buf, buflen, hi) || + mpack_w4(buf, buflen, lo); + } else if (lo > 0xffff) { + /* uint 32 */ + return mpack_w1(buf, buflen, 0xce) || + mpack_w4(buf, buflen, lo); + } else if (lo > 0xff) { + /* uint 16 */ + return mpack_w1(buf, buflen, 0xcd) || + mpack_w2(buf, buflen, lo); + } else if (lo > 0x7f) { + /* uint 8 */ + return mpack_w1(buf, buflen, 0xcc) || + mpack_w1(buf, buflen, lo); + } else { + return mpack_w1(buf, buflen, lo); + } +} + +static int mpack_wnint(char **buf, size_t *buflen, mpack_value_t val) +{ + mpack_uint32_t hi = val.hi; + mpack_uint32_t lo = val.lo; + + if (lo < 0x80000000) { + /* int 64 */ + return mpack_w1(buf, buflen, 0xd3) || + mpack_w4(buf, buflen, hi) || + mpack_w4(buf, buflen, lo); + } else if (lo < 0xffff7fff) { + /* int 32 */ + return mpack_w1(buf, buflen, 0xd2) || + mpack_w4(buf, buflen, lo); + } else if (lo < 0xffffff7f) { + /* int 16 */ + return mpack_w1(buf, buflen, 0xd1) || + mpack_w2(buf, buflen, lo); + } else if (lo < 0xffffffe0) { + /* int 8 */ + return mpack_w1(buf, buflen, 0xd0) || + mpack_w1(buf, buflen, lo); + } else { + /* negative fixint */ + return mpack_w1(buf, buflen, (mpack_uint32_t)(0x100 + lo)); + } +} + +static int mpack_wfloat(char **buf, size_t *buflen, + const mpack_token_t *tok) +{ + if (tok->length == 4) { + return mpack_w1(buf, buflen, 0xca) || + mpack_w4(buf, buflen, tok->data.value.lo); + } else if (tok->length == 8) { + return mpack_w1(buf, buflen, 0xcb) || + mpack_w4(buf, buflen, tok->data.value.hi) || + mpack_w4(buf, buflen, tok->data.value.lo); + } else { + return MPACK_ERROR; + } +} + +static int mpack_wstr(char **buf, size_t *buflen, mpack_uint32_t len) +{ + if (len < 0x20) { + return mpack_w1(buf, buflen, 0xa0 | len); + } else if (len < 0x100) { + return mpack_w1(buf, buflen, 0xd9) || + mpack_w1(buf, buflen, len); + } else if (len < 0x10000) { + return mpack_w1(buf, buflen, 0xda) || + mpack_w2(buf, buflen, len); + } else { + return mpack_w1(buf, buflen, 0xdb) || + mpack_w4(buf, buflen, len); + } +} + +static int mpack_wbin(char **buf, size_t *buflen, mpack_uint32_t len) +{ + if (len < 0x100) { + return mpack_w1(buf, buflen, 0xc4) || + mpack_w1(buf, buflen, len); + } else if (len < 0x10000) { + return mpack_w1(buf, buflen, 0xc5) || + mpack_w2(buf, buflen, len); + } else { + return mpack_w1(buf, buflen, 0xc6) || + mpack_w4(buf, buflen, len); + } +} + +static int mpack_wext(char **buf, size_t *buflen, int type, + mpack_uint32_t len) +{ + mpack_uint32_t t; + assert(type >= 0 && type < 0x80); + t = (mpack_uint32_t)type; + switch (len) { + case 1: mpack_w1(buf, buflen, 0xd4); return mpack_w1(buf, buflen, t); + case 2: mpack_w1(buf, buflen, 0xd5); return mpack_w1(buf, buflen, t); + case 4: mpack_w1(buf, buflen, 0xd6); return mpack_w1(buf, buflen, t); + case 8: mpack_w1(buf, buflen, 0xd7); return mpack_w1(buf, buflen, t); + case 16: mpack_w1(buf, buflen, 0xd8); return mpack_w1(buf, buflen, t); + default: + if (len < 0x100) { + return mpack_w1(buf, buflen, 0xc7) || + mpack_w1(buf, buflen, len) || + mpack_w1(buf, buflen, t); + } else if (len < 0x10000) { + return mpack_w1(buf, buflen, 0xc8) || + mpack_w2(buf, buflen, len) || + mpack_w1(buf, buflen, t); + } else { + return mpack_w1(buf, buflen, 0xc9) || + mpack_w4(buf, buflen, len) || + mpack_w1(buf, buflen, t); + } + } +} + +static int mpack_warray(char **buf, size_t *buflen, mpack_uint32_t len) +{ + if (len < 0x10) { + return mpack_w1(buf, buflen, 0x90 | len); + } else if (len < 0x10000) { + return mpack_w1(buf, buflen, 0xdc) || + mpack_w2(buf, buflen, len); + } else { + return mpack_w1(buf, buflen, 0xdd) || + mpack_w4(buf, buflen, len); + } +} + +static int mpack_wmap(char **buf, size_t *buflen, mpack_uint32_t len) +{ + if (len < 0x10) { + return mpack_w1(buf, buflen, 0x80 | len); + } else if (len < 0x10000) { + return mpack_w1(buf, buflen, 0xde) || + mpack_w2(buf, buflen, len); + } else { + return mpack_w1(buf, buflen, 0xdf) || + mpack_w4(buf, buflen, len); + } +} + +static int mpack_w1(char **b, size_t *bl, mpack_uint32_t v) +{ + (*bl)--; + *(*b)++ = (char)(v & 0xff); + return MPACK_OK; +} + +static int mpack_w2(char **b, size_t *bl, mpack_uint32_t v) +{ + *bl -= 2; + *(*b)++ = (char)((v >> 8) & 0xff); + *(*b)++ = (char)(v & 0xff); + return MPACK_OK; +} + +static int mpack_w4(char **b, size_t *bl, mpack_uint32_t v) +{ + *bl -= 4; + *(*b)++ = (char)((v >> 24) & 0xff); + *(*b)++ = (char)((v >> 16) & 0xff); + *(*b)++ = (char)((v >> 8) & 0xff); + *(*b)++ = (char)(v & 0xff); + return MPACK_OK; +} + +static int mpack_value(mpack_token_type_t type, mpack_uint32_t length, + mpack_value_t value, mpack_token_t *tok) +{ + tok->type = type; + tok->length = length; + tok->data.value = value; + return MPACK_OK; +} + +static int mpack_blob(mpack_token_type_t type, mpack_uint32_t length, + int ext_type, mpack_token_t *tok) +{ + tok->type = type; + tok->length = length; + tok->data.ext_type = ext_type; + return MPACK_OK; +} + +static mpack_value_t mpack_byte(unsigned char byte) +{ + mpack_value_t rv; + rv.lo = byte; + rv.hi = 0; + return rv; +} diff --git a/src/mpack/mpack_core.h b/src/mpack/mpack_core.h new file mode 100644 index 0000000000..9edd13c41e --- /dev/null +++ b/src/mpack/mpack_core.h @@ -0,0 +1,87 @@ +#ifndef MPACK_CORE_H +#define MPACK_CORE_H + +#ifndef MPACK_API +# define MPACK_API extern +#endif + +#include <assert.h> +#include <limits.h> +#include <stddef.h> + +#ifdef __GNUC__ +# define FPURE __attribute__((const)) +# define FNONULL __attribute__((nonnull)) +# define FNONULL_ARG(x) __attribute__((nonnull x)) +# define FUNUSED __attribute__((unused)) +#else +# define FPURE +# define FNONULL +# define FNONULL_ARG(x) +# define FUNUSED +#endif + +#if UINT_MAX == 0xffffffff +typedef int mpack_sint32_t; +typedef unsigned int mpack_uint32_t; +#elif ULONG_MAX == 0xffffffff +typedef long mpack_sint32_t; +typedef unsigned long mpack_uint32_t; +#else +# error "can't find unsigned 32-bit integer type" +#endif + +typedef struct mpack_value_s { + mpack_uint32_t lo, hi; +} mpack_value_t; + + +enum { + MPACK_OK = 0, + MPACK_EOF = 1, + MPACK_ERROR = 2 +}; + +#define MPACK_MAX_TOKEN_LEN 9 /* 64-bit ints/floats plus type code */ + +typedef enum { + MPACK_TOKEN_NIL = 1, + MPACK_TOKEN_BOOLEAN = 2, + MPACK_TOKEN_UINT = 3, + MPACK_TOKEN_SINT = 4, + MPACK_TOKEN_FLOAT = 5, + MPACK_TOKEN_CHUNK = 6, + MPACK_TOKEN_ARRAY = 7, + MPACK_TOKEN_MAP = 8, + MPACK_TOKEN_BIN = 9, + MPACK_TOKEN_STR = 10, + MPACK_TOKEN_EXT = 11 +} mpack_token_type_t; + +typedef struct mpack_token_s { + mpack_token_type_t type; /* Type of token */ + mpack_uint32_t length; /* Byte length for str/bin/ext/chunk/float/int/uint. + Item count for array/map. */ + union { + mpack_value_t value; /* 32-bit parts of primitives (bool,int,float) */ + const char *chunk_ptr; /* Chunk of data from str/bin/ext */ + int ext_type; /* Type field for ext tokens */ + } data; +} mpack_token_t; + +typedef struct mpack_tokbuf_s { + char pending[MPACK_MAX_TOKEN_LEN]; + mpack_token_t pending_tok; + size_t ppos, plen; + mpack_uint32_t passthrough; +} mpack_tokbuf_t; + +#define MPACK_TOKBUF_INITIAL_VALUE { { 0 }, { 0, 0, { { 0, 0 } } }, 0, 0, 0 } + +MPACK_API void mpack_tokbuf_init(mpack_tokbuf_t *tb) FUNUSED FNONULL; +MPACK_API int mpack_read(mpack_tokbuf_t *tb, const char **b, size_t *bl, + mpack_token_t *tok) FUNUSED FNONULL; +MPACK_API int mpack_write(mpack_tokbuf_t *tb, char **b, size_t *bl, + const mpack_token_t *tok) FUNUSED FNONULL; + +#endif /* MPACK_CORE_H */ diff --git a/src/mpack/object.c b/src/mpack/object.c new file mode 100644 index 0000000000..0c7759ee51 --- /dev/null +++ b/src/mpack/object.c @@ -0,0 +1,195 @@ +#include <string.h> + +#include "object.h" + +static int mpack_parser_full(mpack_parser_t *w); +static mpack_node_t *mpack_parser_push(mpack_parser_t *w); +static mpack_node_t *mpack_parser_pop(mpack_parser_t *w); + +MPACK_API void mpack_parser_init(mpack_parser_t *parser, + mpack_uint32_t capacity) +{ + mpack_tokbuf_init(&parser->tokbuf); + parser->data.p = NULL; + parser->capacity = capacity ? capacity : MPACK_MAX_OBJECT_DEPTH; + parser->size = 0; + parser->exiting = 0; + memset(parser->items, 0, sizeof(mpack_node_t) * (parser->capacity + 1)); + parser->items[0].pos = (size_t)-1; + parser->status = 0; +} + +#define MPACK_EXCEPTION_CHECK(parser) \ + do { \ + if (parser->status == MPACK_EXCEPTION) { \ + return MPACK_EXCEPTION; \ + } \ + } while (0) + +#define MPACK_WALK(action) \ + do { \ + mpack_node_t *n; \ + \ + if (parser->exiting) goto exit; \ + if (mpack_parser_full(parser)) return MPACK_NOMEM; \ + n = mpack_parser_push(parser); \ + action; \ + MPACK_EXCEPTION_CHECK(parser); \ + parser->exiting = 1; \ + return MPACK_EOF; \ + \ +exit: \ + parser->exiting = 0; \ + while ((n = mpack_parser_pop(parser))) { \ + exit_cb(parser, n); \ + MPACK_EXCEPTION_CHECK(parser); \ + if (!parser->size) return MPACK_OK; \ + } \ + \ + return MPACK_EOF; \ + } while (0) + +MPACK_API int mpack_parse_tok(mpack_parser_t *parser, mpack_token_t tok, + mpack_walk_cb enter_cb, mpack_walk_cb exit_cb) +{ + MPACK_EXCEPTION_CHECK(parser); + MPACK_WALK({n->tok = tok; enter_cb(parser, n);}); +} + +MPACK_API int mpack_unparse_tok(mpack_parser_t *parser, mpack_token_t *tok, + mpack_walk_cb enter_cb, mpack_walk_cb exit_cb) +{ + MPACK_EXCEPTION_CHECK(parser); + MPACK_WALK({enter_cb(parser, n); *tok = n->tok;}); +} + +MPACK_API int mpack_parse(mpack_parser_t *parser, const char **buf, + size_t *buflen, mpack_walk_cb enter_cb, mpack_walk_cb exit_cb) +{ + int status = MPACK_EOF; + MPACK_EXCEPTION_CHECK(parser); + + while (*buflen && status) { + mpack_token_t tok; + mpack_tokbuf_t *tb = &parser->tokbuf; + const char *buf_save = *buf; + size_t buflen_save = *buflen; + + if ((status = mpack_read(tb, buf, buflen, &tok)) == MPACK_EOF) continue; + else if (status == MPACK_ERROR) goto rollback; + + do { + status = mpack_parse_tok(parser, tok, enter_cb, exit_cb); + MPACK_EXCEPTION_CHECK(parser); + } while (parser->exiting); + + if (status != MPACK_NOMEM) continue; + +rollback: + /* restore buf/buflen so the next call will try to read the same token */ + *buf = buf_save; + *buflen = buflen_save; + break; + } + + return status; +} + +MPACK_API int mpack_unparse(mpack_parser_t *parser, char **buf, size_t *buflen, + mpack_walk_cb enter_cb, mpack_walk_cb exit_cb) +{ + int status = MPACK_EOF; + MPACK_EXCEPTION_CHECK(parser); + + while (*buflen && status) { + int write_status; + mpack_token_t tok; + mpack_tokbuf_t *tb = &parser->tokbuf; + + if (!tb->plen) + parser->status = mpack_unparse_tok(parser, &tok, enter_cb, exit_cb); + + MPACK_EXCEPTION_CHECK(parser); + + status = parser->status; + + if (status == MPACK_NOMEM) + break; + + if (parser->exiting) { + write_status = mpack_write(tb, buf, buflen, &tok); + status = write_status ? write_status : status; + } + } + + return status; +} + +MPACK_API void mpack_parser_copy(mpack_parser_t *dst, mpack_parser_t *src) +{ + mpack_uint32_t i; + mpack_uint32_t dst_capacity = dst->capacity; + assert(src->capacity <= dst_capacity); + /* copy all fields except the stack */ + memcpy(dst, src, sizeof(mpack_one_parser_t) - sizeof(mpack_node_t)); + /* reset capacity */ + dst->capacity = dst_capacity; + /* copy the stack */ + for (i = 0; i <= src->capacity; i++) { + dst->items[i] = src->items[i]; + } +} + +static int mpack_parser_full(mpack_parser_t *parser) +{ + return parser->size == parser->capacity; +} + +static mpack_node_t *mpack_parser_push(mpack_parser_t *parser) +{ + mpack_node_t *top; + assert(parser->size < parser->capacity); + top = parser->items + parser->size + 1; + top->data[0].p = NULL; + top->data[1].p = NULL; + top->pos = 0; + top->key_visited = 0; + /* increase size and invoke callback, passing parent node if any */ + parser->size++; + return top; +} + +static mpack_node_t *mpack_parser_pop(mpack_parser_t *parser) +{ + mpack_node_t *top, *parent; + assert(parser->size); + top = parser->items + parser->size; + + if (top->tok.type > MPACK_TOKEN_CHUNK && top->pos < top->tok.length) { + /* continue processing children */ + return NULL; + } + + parent = MPACK_PARENT_NODE(top); + if (parent) { + /* we use parent->tok.length to keep track of how many children remain. + * update it to reflect the processed node. */ + if (top->tok.type == MPACK_TOKEN_CHUNK) { + parent->pos += top->tok.length; + } else if (parent->tok.type == MPACK_TOKEN_MAP) { + /* maps allow up to 2^32 - 1 pairs, so to allow this many items in a + * 32-bit length variable we use an additional flag to determine if the + * key of a certain position was visited */ + if (parent->key_visited) { + parent->pos++; + } + parent->key_visited = !parent->key_visited; + } else { + parent->pos++; + } + } + + parser->size--; + return top; +} + diff --git a/src/mpack/object.h b/src/mpack/object.h new file mode 100644 index 0000000000..5327e56e18 --- /dev/null +++ b/src/mpack/object.h @@ -0,0 +1,86 @@ +#ifndef MPACK_OBJECT_H +#define MPACK_OBJECT_H + +#include "mpack_core.h" +#include "conv.h" + +#ifndef MPACK_MAX_OBJECT_DEPTH +# define MPACK_MAX_OBJECT_DEPTH 32 +#endif + +#define MPACK_PARENT_NODE(n) (((n) - 1)->pos == (size_t)-1 ? NULL : (n) - 1) + +#define MPACK_THROW(parser) \ + do { \ + parser->status = MPACK_EXCEPTION; \ + return; \ + } while (0) + +enum { + MPACK_EXCEPTION = -1, + MPACK_NOMEM = MPACK_ERROR + 1 +}; + +/* Storing integer in pointers in undefined behavior according to the C + * standard. Define a union type to accomodate arbitrary user data associated + * with nodes(and with requests in rpc.h). */ +typedef union { + void *p; + mpack_uintmax_t u; + mpack_sintmax_t i; + double d; +} mpack_data_t; + +typedef struct mpack_node_s { + mpack_token_t tok; + size_t pos; + /* flag to determine if the key was visited when traversing a map */ + int key_visited; + /* allow 2 instances mpack_data_t per node. the reason is that when + * serializing, the user may need to keep track of traversal state besides the + * parent node reference */ + mpack_data_t data[2]; +} mpack_node_t; + +#define MPACK_PARSER_STRUCT(c) \ + struct { \ + mpack_data_t data; \ + mpack_uint32_t size, capacity; \ + int status; \ + int exiting; \ + mpack_tokbuf_t tokbuf; \ + mpack_node_t items[c + 1]; \ + } + +/* Some compilers warn against anonymous structs: + * https://github.com/libmpack/libmpack/issues/6 */ +typedef MPACK_PARSER_STRUCT(0) mpack_one_parser_t; + +#define MPACK_PARSER_STRUCT_SIZE(c) \ + (sizeof(mpack_node_t) * c + \ + sizeof(mpack_one_parser_t)) + +typedef MPACK_PARSER_STRUCT(MPACK_MAX_OBJECT_DEPTH) mpack_parser_t; +typedef void(*mpack_walk_cb)(mpack_parser_t *w, mpack_node_t *n); + +MPACK_API void mpack_parser_init(mpack_parser_t *p, mpack_uint32_t c) + FUNUSED FNONULL; + +MPACK_API int mpack_parse_tok(mpack_parser_t *walker, mpack_token_t tok, + mpack_walk_cb enter_cb, mpack_walk_cb exit_cb) + FUNUSED FNONULL_ARG((1,3,4)); +MPACK_API int mpack_unparse_tok(mpack_parser_t *walker, mpack_token_t *tok, + mpack_walk_cb enter_cb, mpack_walk_cb exit_cb) + FUNUSED FNONULL_ARG((1,2,3,4)); + +MPACK_API int mpack_parse(mpack_parser_t *parser, const char **b, size_t *bl, + mpack_walk_cb enter_cb, mpack_walk_cb exit_cb) + FUNUSED FNONULL_ARG((1,2,3,4,5)); +MPACK_API int mpack_unparse(mpack_parser_t *parser, char **b, size_t *bl, + mpack_walk_cb enter_cb, mpack_walk_cb exit_cb) + FUNUSED FNONULL_ARG((1,2,3,4,5)); + +MPACK_API void mpack_parser_copy(mpack_parser_t *d, mpack_parser_t *s) + FUNUSED FNONULL; + +#endif /* MPACK_OBJECT_H */ diff --git a/src/mpack/rpc.c b/src/mpack/rpc.c new file mode 100644 index 0000000000..3b2b328065 --- /dev/null +++ b/src/mpack/rpc.c @@ -0,0 +1,331 @@ +#include <string.h> + +#include "rpc.h" + +enum { + MPACK_RPC_RECEIVE_ARRAY = 1, + MPACK_RPC_RECEIVE_TYPE, + MPACK_RPC_RECEIVE_ID +}; + +static mpack_rpc_header_t mpack_rpc_request_hdr(void); +static mpack_rpc_header_t mpack_rpc_reply_hdr(void); +static mpack_rpc_header_t mpack_rpc_notify_hdr(void); +static int mpack_rpc_put(mpack_rpc_session_t *s, mpack_rpc_message_t m); +static int mpack_rpc_pop(mpack_rpc_session_t *s, mpack_rpc_message_t *m); +static void mpack_rpc_reset_hdr(mpack_rpc_header_t *hdr); + +MPACK_API void mpack_rpc_session_init(mpack_rpc_session_t *session, + mpack_uint32_t capacity) +{ + session->capacity = capacity ? capacity : MPACK_RPC_MAX_REQUESTS; + session->request_id = 0; + mpack_tokbuf_init(&session->reader); + mpack_tokbuf_init(&session->writer); + mpack_rpc_reset_hdr(&session->receive); + mpack_rpc_reset_hdr(&session->send); + memset(session->slots, 0, + sizeof(struct mpack_rpc_slot_s) * session->capacity); +} + +MPACK_API int mpack_rpc_receive_tok(mpack_rpc_session_t *session, + mpack_token_t tok, mpack_rpc_message_t *msg) +{ + int type; + + if (session->receive.index == 0) { + if (tok.type != MPACK_TOKEN_ARRAY) + /* not an array */ + return MPACK_RPC_EARRAY; + + if (tok.length < 3 || tok.length > 4) + /* invalid array length */ + return MPACK_RPC_EARRAYL; + + session->receive.toks[0] = tok; + session->receive.index++; + return MPACK_EOF; /* get the type */ + } + + if (session->receive.index == 1) { + + if (tok.type != MPACK_TOKEN_UINT || tok.length > 1 || tok.data.value.lo > 2) + /* invalid type */ + return MPACK_RPC_ETYPE; + + if (tok.data.value.lo < 2 && session->receive.toks[0].length != 4) + /* request or response with array length != 4 */ + return MPACK_RPC_EARRAYL; + + if (tok.data.value.lo == 2 && session->receive.toks[0].length != 3) + /* notification with array length != 3 */ + return MPACK_RPC_EARRAYL; + + session->receive.toks[1] = tok; + session->receive.index++; + + if (tok.data.value.lo < 2) return MPACK_EOF; + + type = MPACK_RPC_NOTIFICATION; + goto end; + } + + assert(session->receive.index == 2); + + if (tok.type != MPACK_TOKEN_UINT || tok.length > 4) + /* invalid request/response id */ + return MPACK_RPC_EMSGID; + + msg->id = tok.data.value.lo; + msg->data.p = NULL; + type = (int)session->receive.toks[1].data.value.lo + MPACK_RPC_REQUEST; + + if (type == MPACK_RPC_RESPONSE && !mpack_rpc_pop(session, msg)) + /* response with invalid id */ + return MPACK_RPC_ERESPID; + +end: + mpack_rpc_reset_hdr(&session->receive); + return type; +} + +MPACK_API int mpack_rpc_request_tok(mpack_rpc_session_t *session, + mpack_token_t *tok, mpack_data_t data) +{ + if (session->send.index == 0) { + int status; + mpack_rpc_message_t msg; + do { + msg.id = session->request_id; + msg.data = data; + session->send = mpack_rpc_request_hdr(); + session->send.toks[2].type = MPACK_TOKEN_UINT; + session->send.toks[2].data.value.lo = msg.id; + session->send.toks[2].data.value.hi = 0; + *tok = session->send.toks[0]; + status = mpack_rpc_put(session, msg); + if (status == -1) return MPACK_NOMEM; + session->request_id = (session->request_id + 1) % 0xffffffff; + } while (!status); + session->send.index++; + return MPACK_EOF; + } + + if (session->send.index == 1) { + *tok = session->send.toks[1]; + session->send.index++; + return MPACK_EOF; + } + + assert(session->send.index == 2); + *tok = session->send.toks[2]; + mpack_rpc_reset_hdr(&session->send); + return MPACK_OK; +} + +MPACK_API int mpack_rpc_reply_tok(mpack_rpc_session_t *session, + mpack_token_t *tok, mpack_uint32_t id) +{ + if (session->send.index == 0) { + session->send = mpack_rpc_reply_hdr(); + session->send.toks[2].type = MPACK_TOKEN_UINT; + session->send.toks[2].data.value.lo = id; + session->send.toks[2].data.value.hi = 0; + *tok = session->send.toks[0]; + session->send.index++; + return MPACK_EOF; + } + + if (session->send.index == 1) { + *tok = session->send.toks[1]; + session->send.index++; + return MPACK_EOF; + } + + assert(session->send.index == 2); + *tok = session->send.toks[2]; + mpack_rpc_reset_hdr(&session->send); + return MPACK_OK; +} + +MPACK_API int mpack_rpc_notify_tok(mpack_rpc_session_t *session, + mpack_token_t *tok) +{ + if (session->send.index == 0) { + session->send = mpack_rpc_notify_hdr(); + *tok = session->send.toks[0]; + session->send.index++; + return MPACK_EOF; + } + + assert(session->send.index == 1); + *tok = session->send.toks[1]; + mpack_rpc_reset_hdr(&session->send); + return MPACK_OK; +} + +MPACK_API int mpack_rpc_receive(mpack_rpc_session_t *session, const char **buf, + size_t *buflen, mpack_rpc_message_t *msg) +{ + int status; + + do { + mpack_token_t tok; + status = mpack_read(&session->reader, buf, buflen, &tok); + if (status) break; + status = mpack_rpc_receive_tok(session, tok, msg); + if (status >= MPACK_RPC_REQUEST) break; + } while (*buflen); + + return status; +} + +MPACK_API int mpack_rpc_request(mpack_rpc_session_t *session, char **buf, + size_t *buflen, mpack_data_t data) +{ + int status = MPACK_EOF; + + while (status && *buflen) { + int write_status; + mpack_token_t tok; + if (!session->writer.plen) { + status = mpack_rpc_request_tok(session, &tok, data); + } + if (status == MPACK_NOMEM) break; + write_status = mpack_write(&session->writer, buf, buflen, &tok); + status = write_status ? write_status : status; + } + + return status; +} + +MPACK_API int mpack_rpc_reply(mpack_rpc_session_t *session, char **buf, + size_t *buflen, mpack_uint32_t id) +{ + int status = MPACK_EOF; + + while (status && *buflen) { + int write_status; + mpack_token_t tok; + if (!session->writer.plen) { + status = mpack_rpc_reply_tok(session, &tok, id); + } + write_status = mpack_write(&session->writer, buf, buflen, &tok); + status = write_status ? write_status : status; + } + + return status; +} + +MPACK_API int mpack_rpc_notify(mpack_rpc_session_t *session, char **buf, + size_t *buflen) +{ + int status = MPACK_EOF; + + while (status && *buflen) { + int write_status; + mpack_token_t tok; + if (!session->writer.plen) { + status = mpack_rpc_notify_tok(session, &tok); + } + write_status = mpack_write(&session->writer, buf, buflen, &tok); + status = write_status ? write_status : status; + } + + return status; +} + +MPACK_API void mpack_rpc_session_copy(mpack_rpc_session_t *dst, + mpack_rpc_session_t *src) +{ + mpack_uint32_t i; + mpack_uint32_t dst_capacity = dst->capacity; + assert(src->capacity <= dst_capacity); + /* copy all fields except slots */ + memcpy(dst, src, sizeof(mpack_rpc_one_session_t) - + sizeof(struct mpack_rpc_slot_s)); + /* reset capacity */ + dst->capacity = dst_capacity; + /* reinsert requests */ + memset(dst->slots, 0, sizeof(struct mpack_rpc_slot_s) * dst->capacity); + for (i = 0; i < src->capacity; i++) { + if (src->slots[i].used) mpack_rpc_put(dst, src->slots[i].msg); + } +} + +static mpack_rpc_header_t mpack_rpc_request_hdr(void) +{ + mpack_rpc_header_t hdr; + hdr.index = 0; + hdr.toks[0].type = MPACK_TOKEN_ARRAY; + hdr.toks[0].length = 4; + hdr.toks[1].type = MPACK_TOKEN_UINT; + hdr.toks[1].data.value.lo = 0; + hdr.toks[1].data.value.hi = 0; + return hdr; +} + +static mpack_rpc_header_t mpack_rpc_reply_hdr(void) +{ + mpack_rpc_header_t hdr = mpack_rpc_request_hdr(); + hdr.toks[1].data.value.lo = 1; + hdr.toks[1].data.value.hi = 0; + return hdr; +} + +static mpack_rpc_header_t mpack_rpc_notify_hdr(void) +{ + mpack_rpc_header_t hdr = mpack_rpc_request_hdr(); + hdr.toks[0].length = 3; + hdr.toks[1].data.value.lo = 2; + hdr.toks[1].data.value.hi = 0; + return hdr; +} + +static int mpack_rpc_put(mpack_rpc_session_t *session, mpack_rpc_message_t msg) +{ + struct mpack_rpc_slot_s *slot = NULL; + mpack_uint32_t i; + mpack_uint32_t hash = msg.id % session->capacity; + + for (i = 0; i < session->capacity; i++) { + if (!session->slots[hash].used || session->slots[hash].msg.id == msg.id) { + slot = session->slots + hash; + break; + } + hash = hash > 0 ? hash - 1 : session->capacity - 1; + } + + if (!slot) return -1; /* no space */ + if (slot->msg.id == msg.id && slot->used) return 0; /* duplicate key */ + slot->msg = msg; + slot->used = 1; + return 1; +} + +static int mpack_rpc_pop(mpack_rpc_session_t *session, mpack_rpc_message_t *msg) +{ + struct mpack_rpc_slot_s *slot = NULL; + mpack_uint32_t i; + mpack_uint32_t hash = msg->id % session->capacity; + + for (i = 0; i < session->capacity; i++) { + if (session->slots[hash].used && session->slots[hash].msg.id == msg->id) { + slot = session->slots + hash; + break; + } + hash = hash > 0 ? hash - 1 : session->capacity - 1; + } + + if (!slot) return 0; + + *msg = slot->msg; + slot->used = 0; + + return 1; +} + +static void mpack_rpc_reset_hdr(mpack_rpc_header_t *hdr) +{ + hdr->index = 0; +} diff --git a/src/mpack/rpc.h b/src/mpack/rpc.h new file mode 100644 index 0000000000..c1e8d656b5 --- /dev/null +++ b/src/mpack/rpc.h @@ -0,0 +1,83 @@ +#ifndef MPACK_RPC_H +#define MPACK_RPC_H + +#include "mpack_core.h" +#include "object.h" + +#ifndef MPACK_RPC_MAX_REQUESTS +# define MPACK_RPC_MAX_REQUESTS 32 +#endif + +enum { + MPACK_RPC_REQUEST = MPACK_NOMEM + 1, + MPACK_RPC_RESPONSE, + MPACK_RPC_NOTIFICATION, + MPACK_RPC_ERROR +}; + +enum { + MPACK_RPC_EARRAY = MPACK_RPC_ERROR, + MPACK_RPC_EARRAYL, + MPACK_RPC_ETYPE, + MPACK_RPC_EMSGID, + MPACK_RPC_ERESPID +}; + +typedef struct mpack_rpc_header_s { + mpack_token_t toks[3]; + int index; +} mpack_rpc_header_t; + +typedef struct mpack_rpc_message_s { + mpack_uint32_t id; + mpack_data_t data; +} mpack_rpc_message_t; + +struct mpack_rpc_slot_s { + int used; + mpack_rpc_message_t msg; +}; + +#define MPACK_RPC_SESSION_STRUCT(c) \ + struct { \ + mpack_tokbuf_t reader, writer; \ + mpack_rpc_header_t receive, send; \ + mpack_uint32_t request_id, capacity; \ + struct mpack_rpc_slot_s slots[c]; \ + } + +/* Some compilers warn against anonymous structs: + * https://github.com/libmpack/libmpack/issues/6 */ +typedef MPACK_RPC_SESSION_STRUCT(1) mpack_rpc_one_session_t; + +#define MPACK_RPC_SESSION_STRUCT_SIZE(c) \ + (sizeof(struct mpack_rpc_slot_s) * (c - 1) + \ + sizeof(mpack_rpc_one_session_t)) + +typedef MPACK_RPC_SESSION_STRUCT(MPACK_RPC_MAX_REQUESTS) mpack_rpc_session_t; + +MPACK_API void mpack_rpc_session_init(mpack_rpc_session_t *s, mpack_uint32_t c) + FUNUSED FNONULL; + +MPACK_API int mpack_rpc_receive_tok(mpack_rpc_session_t *s, mpack_token_t t, + mpack_rpc_message_t *msg) FUNUSED FNONULL; +MPACK_API int mpack_rpc_request_tok(mpack_rpc_session_t *s, mpack_token_t *t, + mpack_data_t d) FUNUSED FNONULL_ARG((1,2)); +MPACK_API int mpack_rpc_reply_tok(mpack_rpc_session_t *s, mpack_token_t *t, + mpack_uint32_t i) FUNUSED FNONULL; +MPACK_API int mpack_rpc_notify_tok(mpack_rpc_session_t *s, mpack_token_t *t) + FUNUSED FNONULL; + +MPACK_API int mpack_rpc_receive(mpack_rpc_session_t *s, const char **b, + size_t *bl, mpack_rpc_message_t *m) FUNUSED FNONULL; +MPACK_API int mpack_rpc_request(mpack_rpc_session_t *s, char **b, size_t *bl, + mpack_data_t d) FUNUSED FNONULL_ARG((1,2,3)); +MPACK_API int mpack_rpc_reply(mpack_rpc_session_t *s, char **b, size_t *bl, + mpack_uint32_t i) FNONULL FUNUSED; +MPACK_API int mpack_rpc_notify(mpack_rpc_session_t *s, char **b, size_t *bl) + FNONULL FUNUSED; + +MPACK_API void mpack_rpc_session_copy(mpack_rpc_session_t *d, + mpack_rpc_session_t *s) FUNUSED FNONULL; + +#endif /* MPACK_RPC_H */ diff --git a/src/nvim/CMakeLists.txt b/src/nvim/CMakeLists.txt index 4a698052ee..331ab16dd7 100644 --- a/src/nvim/CMakeLists.txt +++ b/src/nvim/CMakeLists.txt @@ -87,8 +87,8 @@ file(MAKE_DIRECTORY ${LINT_SUPPRESSES_ROOT}/src) file(GLOB NVIM_SOURCES *.c) file(GLOB NVIM_HEADERS *.h) -file(GLOB XDIFF_SOURCES ../xdiff/*.c) -file(GLOB XDIFF_HEADERS ../xdiff/*.h) +file(GLOB EXTERNAL_SOURCES ../xdiff/*.c ../mpack/*.c) +file(GLOB EXTERNAL_HEADERS ../xdiff/*.h ../mpack/*.h) foreach(subdir os @@ -171,8 +171,8 @@ foreach(sfile ${CONV_SOURCES}) message(FATAL_ERROR "${sfile} doesn't exist (it was added to CONV_SOURCES)") endif() endforeach() -# xdiff: inlined external project, we don't maintain it. #9306 -list(APPEND CONV_SOURCES ${XDIFF_SOURCES}) +# xdiff, mpack: inlined external project, we don't maintain it. #9306 +list(APPEND CONV_SOURCES ${EXTERNAL_SOURCES}) if(NOT MSVC) set_source_files_properties( @@ -471,7 +471,7 @@ endif() add_executable(nvim ${NVIM_GENERATED_FOR_SOURCES} ${NVIM_GENERATED_FOR_HEADERS} ${NVIM_GENERATED_SOURCES} ${NVIM_SOURCES} ${NVIM_HEADERS} - ${XDIFF_SOURCES} ${XDIFF_HEADERS}) + ${EXTERNAL_SOURCES} ${EXTERNAL_HEADERS}) target_link_libraries(nvim ${NVIM_EXEC_LINK_LIBRARIES}) install_helper(TARGETS nvim) @@ -602,7 +602,7 @@ add_library( EXCLUDE_FROM_ALL ${NVIM_SOURCES} ${NVIM_GENERATED_SOURCES} ${NVIM_HEADERS} ${NVIM_GENERATED_FOR_SOURCES} ${NVIM_GENERATED_FOR_HEADERS} - ${XDIFF_SOURCES} ${XDIFF_HEADERS} + ${EXTERNAL_SOURCES} ${EXTERNAL_HEADERS} ) set_property(TARGET libnvim APPEND PROPERTY INCLUDE_DIRECTORIES ${LUA_PREFERRED_INCLUDE_DIRS}) @@ -632,7 +632,7 @@ else() EXCLUDE_FROM_ALL ${NVIM_SOURCES} ${NVIM_GENERATED_SOURCES} ${NVIM_HEADERS} ${NVIM_GENERATED_FOR_SOURCES} ${NVIM_GENERATED_FOR_HEADERS} - ${XDIFF_SOURCES} ${XDIFF_HEADERS} + ${EXTERNAL_SOURCES} ${EXTERNAL_HEADERS} ${UNIT_TEST_FIXTURES} ) target_link_libraries(nvim-test ${NVIM_TEST_LINK_LIBRARIES}) diff --git a/src/nvim/getchar.c b/src/nvim/getchar.c index 28f58e2c34..11e3b9bc2d 100644 --- a/src/nvim/getchar.c +++ b/src/nvim/getchar.c @@ -1563,8 +1563,8 @@ int vgetc(void) */ may_garbage_collect = false; - // Exec lua callbacks for on_keystroke - nlua_execute_log_keystroke(c); + // Execute Lua on_key callbacks. + nlua_execute_on_key(c); return c; } diff --git a/src/nvim/lua/executor.c b/src/nvim/lua/executor.c index 5cd9894f9d..d071203db1 100644 --- a/src/nvim/lua/executor.c +++ b/src/nvim/lua/executor.c @@ -33,6 +33,7 @@ #include "nvim/eval/userfunc.h" #include "nvim/event/time.h" #include "nvim/event/loop.h" +#include "mpack/lmpack.h" #include "nvim/os/os.h" @@ -506,6 +507,8 @@ static int nlua_state_init(lua_State *const lstate) FUNC_ATTR_NONNULL_ALL lua_setfield(lstate, -2, "__tostring"); lua_setmetatable(lstate, -2); nlua_nil_ref = nlua_ref(lstate, -1); + lua_pushvalue(lstate, -1); + lua_setfield(lstate, LUA_REGISTRYINDEX, "mpack.NIL"); lua_setfield(lstate, -2, "NIL"); // vim._empty_dict_mt @@ -513,8 +516,23 @@ static int nlua_state_init(lua_State *const lstate) FUNC_ATTR_NONNULL_ALL lua_pushcfunction(lstate, &nlua_empty_dict_tostring); lua_setfield(lstate, -2, "__tostring"); nlua_empty_dict_ref = nlua_ref(lstate, -1); + lua_pushvalue(lstate, -1); + lua_setfield(lstate, LUA_REGISTRYINDEX, "mpack.empty_dict"); lua_setfield(lstate, -2, "_empty_dict_mt"); + // vim.mpack + luaopen_mpack(lstate); + lua_pushvalue(lstate, -1); + lua_setfield(lstate, -3, "mpack"); + + // package.loaded.mpack = vim.mpack + // otherwise luv will be reinitialized when require'mpack' + lua_getglobal(lstate, "package"); + lua_getfield(lstate, -1, "loaded"); + lua_pushvalue(lstate, -3); + lua_setfield(lstate, -2, "mpack"); + lua_pop(lstate, 3); + // internal vim._treesitter... API nlua_add_treesitter(lstate); @@ -1499,7 +1517,7 @@ int nlua_expand_pat(expand_T *xp, lua_getfield(lstate, -1, "_expand_pat"); luaL_checktype(lstate, -1, LUA_TFUNCTION); - // [ vim, vim._log_keystroke, buf ] + // [ vim, vim._on_key, buf ] lua_pushlstring(lstate, (const char *)pat, STRLEN(pat)); if (lua_pcall(lstate, 1, 2, 0) != 0) { @@ -1773,7 +1791,7 @@ char_u *nlua_register_table_as_callable(typval_T *const arg) return name; } -void nlua_execute_log_keystroke(int c) +void nlua_execute_on_key(int c) { char_u buf[NUMBUFLEN]; size_t buf_len = special_to_buf(c, mod_mask, false, buf); @@ -1787,17 +1805,17 @@ void nlua_execute_log_keystroke(int c) // [ vim ] lua_getglobal(lstate, "vim"); - // [ vim, vim._log_keystroke ] - lua_getfield(lstate, -1, "_log_keystroke"); + // [ vim, vim._on_key] + lua_getfield(lstate, -1, "_on_key"); luaL_checktype(lstate, -1, LUA_TFUNCTION); - // [ vim, vim._log_keystroke, buf ] + // [ vim, vim._on_key, buf ] lua_pushlstring(lstate, (const char *)buf, buf_len); if (lua_pcall(lstate, 1, 0, 0)) { nlua_error( lstate, - _("Error executing vim.log_keystroke lua callback: %.*s")); + _("Error executing vim.on_key Lua callback: %.*s")); } // [ vim ] diff --git a/src/nvim/lua/vim.lua b/src/nvim/lua/vim.lua index 34b314b40d..c6bbdee7ad 100644 --- a/src/nvim/lua/vim.lua +++ b/src/nvim/lua/vim.lua @@ -4,6 +4,7 @@ -- 1. runtime/lua/vim/ (the runtime): For "nice to have" features, e.g. the -- `inspect` and `lpeg` modules. -- 2. runtime/lua/vim/shared.lua: Code shared between Nvim and tests. +-- (This will go away if we migrate to nvim as the test-runner.) -- 3. src/nvim/lua/: Compiled-into Nvim itself. -- -- Guideline: "If in doubt, put it in the runtime". @@ -426,26 +427,35 @@ function vim.notify(msg, log_level, _opts) end -local on_keystroke_callbacks = {} +function vim.register_keystroke_callback() + error('vim.register_keystroke_callback is deprecated, instead use: vim.on_key') +end + +local on_key_cbs = {} ---- Register a lua {fn} with an {id} to be run after every keystroke. +--- Adds Lua function {fn} with namespace id {ns_id} as a listener to every, +--- yes every, input key. --- ----@param fn function: Function to call. It should take one argument, which is a string. ---- The string will contain the literal keys typed. ---- See |i_CTRL-V| +--- The Nvim command-line option |-w| is related but does not support callbacks +--- and cannot be toggled dynamically. --- +---@param fn function: Callback function. It should take one string argument. +--- On each key press, Nvim passes the key char to fn(). |i_CTRL-V| --- If {fn} is nil, it removes the callback for the associated {ns_id} ----@param ns_id number? Namespace ID. If not passed or 0, will generate and return a new ---- namespace ID from |nvim_create_namesapce()| +---@param ns_id number? Namespace ID. If nil or 0, generates and returns a new +--- |nvim_create_namesapce()| id. --- ----@return number Namespace ID associated with {fn} +---@return number Namespace id associated with {fn}. Or count of all callbacks +---if on_key() is called without arguments. --- ----@note {fn} will be automatically removed if an error occurs while calling. ---- This is to prevent the annoying situation of every keystroke erroring ---- while trying to remove a broken callback. ----@note {fn} will not be cleared from |nvim_buf_clear_namespace()| ----@note {fn} will receive the keystrokes after mappings have been evaluated -function vim.register_keystroke_callback(fn, ns_id) +---@note {fn} will be removed if an error occurs while calling. +---@note {fn} will not be cleared by |nvim_buf_clear_namespace()| +---@note {fn} will receive the keys after mappings have been evaluated +function vim.on_key(fn, ns_id) + if fn == nil and ns_id == nil then + return #on_key_cbs + end + vim.validate { fn = { fn, 'c', true}, ns_id = { ns_id, 'n', true } @@ -455,20 +465,19 @@ function vim.register_keystroke_callback(fn, ns_id) ns_id = vim.api.nvim_create_namespace('') end - on_keystroke_callbacks[ns_id] = fn + on_key_cbs[ns_id] = fn return ns_id end ---- Function that executes the keystroke callbacks. +--- Executes the on_key callbacks. ---@private -function vim._log_keystroke(char) +function vim._on_key(char) local failed_ns_ids = {} local failed_messages = {} - for k, v in pairs(on_keystroke_callbacks) do + for k, v in pairs(on_key_cbs) do local ok, err_msg = pcall(v, char) if not ok then - vim.register_keystroke_callback(nil, k) - + vim.on_key(nil, k) table.insert(failed_ns_ids, k) table.insert(failed_messages, err_msg) end @@ -476,7 +485,7 @@ function vim._log_keystroke(char) if failed_ns_ids[1] then error(string.format( - "Error executing 'on_keystroke' with ns_ids of '%s'\n With messages: %s", + "Error executing 'on_key' with ns_ids '%s'\n Messages: %s", table.concat(failed_ns_ids, ", "), table.concat(failed_messages, "\n"))) end diff --git a/src/nvim/main.c b/src/nvim/main.c index b73f5aad76..716434f32e 100644 --- a/src/nvim/main.c +++ b/src/nvim/main.c @@ -334,7 +334,7 @@ int main(int argc, char **argv) // prepare screen now, so external UIs can display messages starting = NO_BUFFERS; screenclear(); - TIME_MSG("initialized screen early for UI"); + TIME_MSG("init screen for UI"); } init_default_mappings(); // Default mappings. diff --git a/src/nvim/mouse.c b/src/nvim/mouse.c index 6c25525936..c599f4ea97 100644 --- a/src/nvim/mouse.c +++ b/src/nvim/mouse.c @@ -723,14 +723,20 @@ int mouse_check_fold(void) int click_row = mouse_row; int click_col = mouse_col; int mouse_char = ' '; + int max_row = Rows; + int max_col = Columns; + int multigrid = ui_has(kUIMultigrid); win_T *wp; wp = mouse_find_win(&click_grid, &click_row, &click_col); + if (wp && multigrid) { + max_row = wp->w_grid_alloc.Rows; + max_col = wp->w_grid_alloc.Columns; + } - if (wp && mouse_row >= 0 && mouse_row < Rows - && mouse_col >= 0 && mouse_col <= Columns) { - int multigrid = ui_has(kUIMultigrid); + if (wp && mouse_row >= 0 && mouse_row < max_row + && mouse_col >= 0 && mouse_col < max_col) { ScreenGrid *gp = multigrid ? &wp->w_grid_alloc : &default_grid; int fdc = win_fdccol_count(wp); int row = multigrid && mouse_grid == 0 ? click_row : mouse_row; diff --git a/src/nvim/po/sr.po b/src/nvim/po/sr.po index 1450ab5164..d34c1c3100 100644 --- a/src/nvim/po/sr.po +++ b/src/nvim/po/sr.po @@ -2,7 +2,7 @@ # # Do ":help uganda" in Vim to read copying and usage conditions. # Do ":help credits" in Vim to see a list of people who contributed. -# Copyright (C) 2017 +# Copyright (C) 2021 # This file is distributed under the same license as the Vim package. # FIRST AUTHOR Ivan Pešić <ivan.pesic@gmail.com>, 2017. # @@ -10,8 +10,8 @@ msgid "" msgstr "" "Project-Id-Version: Vim(Serbian)\n" "Report-Msgid-Bugs-To: \n" -"POT-Creation-Date: 2021-02-14 01:49+0400\n" -"PO-Revision-Date: 2021-02-14 01:54+0400\n" +"POT-Creation-Date: 2021-06-13 13:16+0400\n" +"PO-Revision-Date: 2021-06-13 13:50+0400\n" "Last-Translator: Ivan Pešić <ivan.pesic@gmail.com>\n" "Language-Team: Serbian\n" "Language: sr\n" @@ -97,6 +97,9 @@ msgstr "Извршавање %s" msgid "autocommand %s" msgstr "аутокоманда %s" +msgid "E972: Blob value does not have the right number of bytes" +msgstr "E972: Блоб вредност нема одговарајући број бајтова" + msgid "E831: bf_key_init() called with empty password" msgstr "E831: bf_key_init() је позвана са празном лозинком" @@ -866,7 +869,7 @@ msgid "E976: using Blob as a String" msgstr "E976: коришћење Blob као String" msgid "E908: using an invalid value as a String" -msgstr "E908: користи се недозвољена вредност као String" +msgstr "E908: Користи се неважећа вредност као Стринг: %s" msgid "E698: variable nested too deep for making a copy" msgstr "E698: променљива је предубоко угњеждена да би се направила копија" @@ -913,7 +916,7 @@ msgid "E928: String required" msgstr "E928: Захтева се String" msgid "E808: Number or Float required" -msgstr "E808: Захтева се Number или Float" +msgstr "E808: Захтева се Број или Покретни" msgid "add() argument" msgstr "add() аргумент" @@ -1130,6 +1133,9 @@ msgstr "" "\n" "# Преградне линије, копиране дословно:\n" +msgid "E503: \"%s\" is not a file or writable device" +msgstr "E503: „%s” није фајл или уређај на који може да се уписује" + msgid "Save As" msgstr "Сачувај као" @@ -4635,12 +4641,12 @@ msgstr "E531: Користите \":gui\" да покренете GUI" msgid "E589: 'backupext' and 'patchmode' are equal" msgstr "E589: 'backupext' и 'patchmode' су истоветни" -msgid "E834: Conflicts with value of 'listchars'" -msgstr "E834: У конфликту са вредношћу 'listchars'" - msgid "E835: Conflicts with value of 'fillchars'" msgstr "E835: У конфликту са вредношћу 'fillchars'" +msgid "E834: Conflicts with value of 'listchars'" +msgstr "E834: У конфликту са вредношћу 'listchars'" + msgid "E617: Cannot be changed in the GTK+ 2 GUI" msgstr "E617: Не може да се промени у GTK+ 2 GUI" @@ -5064,7 +5070,7 @@ msgstr "E554: Синтаксна грешка у %s{...}" #, c-format msgid "E888: (NFA regexp) cannot repeat %s" -msgstr "E888: (NFA regexp) не може да се понови %s" +msgstr "E888: (НКА регуларни израз) не може да се понови %s" msgid "" "E864: \\%#= can only be followed by 0, 1, or 2. The automatic engine will be " @@ -5130,15 +5136,15 @@ msgid "External submatches:\n" msgstr "Спољна подпоклапања:\n" msgid "E865: (NFA) Regexp end encountered prematurely" -msgstr "E865: Крај (NFA) Regexp израза је достигнут прерано" +msgstr "E865: (НКА) прерано је достигнут крај регуларног израза" #, c-format msgid "E866: (NFA regexp) Misplaced %c" -msgstr "E866: (NFA regexp) %c је на погрешном месту" +msgstr "E866: (НКА регуларни израз) %c је на погрешном месту" #, c-format msgid "E877: (NFA regexp) Invalid character class: %d" -msgstr "E877: (NFA regexp) Неважећа карактер класа: %d" +msgstr "E877: (НКА регуларни израз) Неважећа карактер класа: %d" #, c-format msgid "E867: (NFA) Unknown operator '\\z%c'" @@ -6422,6 +6428,13 @@ msgstr "E853: Име аргумента је дуплирано: %s" msgid "E989: Non-default argument follows default argument" msgstr "E989: Неподразумевани аргумент следи иза подразумеваног аргумента" +msgid "E126: Missing :endfunction" +msgstr "E126: Недостаје :endfunction" + +#, c-format +msgid "W22: Text found after :endfunction: %s" +msgstr "W22: Пронађен текст након :endfunction: %s" + #, c-format msgid "E451: Expected }: %s" msgstr "E451: Очекује се }: %s" @@ -6497,17 +6510,6 @@ msgstr "E862: Овде не може да се користи g:" msgid "E932: Closure function should not be at top level: %s" msgstr "E932: Затварајућа функција не би требало да буде на највишем нивоу: %s" -msgid "E126: Missing :endfunction" -msgstr "E126: Недостаје :endfunction" - -#, c-format -msgid "W1001: Text found after :enddef: %s" -msgstr "W1001: Пронађен је текст након :enddef: %s" - -#, c-format -msgid "W22: Text found after :endfunction: %s" -msgstr "W22: Пронађен текст након :endfunction: %s" - #, c-format msgid "E707: Function name conflicts with variable: %s" msgstr "E707: Име функције је у конфликту са променљивом: %s" @@ -6983,8 +6985,8 @@ msgid "E475: Invalid value for argument %s: %s" msgstr "E475: Неважећа вредност за аргумент %s: %s" #, c-format -msgid "E15: Invalid expression: %s" -msgstr "E15: Неважећи израз: %s" +msgid "E15: Invalid expression: \"%s\"" +msgstr "E15: Неважећи израз: „%s”" msgid "E16: Invalid range" msgstr "E16: Неважећи опсег" diff --git a/test/busted/outputHandlers/nvim.lua b/test/busted/outputHandlers/nvim.lua index 5456e9ca98..17154f1aa2 100644 --- a/test/busted/outputHandlers/nvim.lua +++ b/test/busted/outputHandlers/nvim.lua @@ -1,15 +1,9 @@ local pretty = require 'pl.pretty' local global_helpers = require('test.helpers') -local colors - -local isWindows = package.config:sub(1,1) == '\\' - -if isWindows then - colors = setmetatable({}, {__index = function() return function(s) return s end end}) -else - colors = require 'term.colors' -end +-- Colors are disabled. #15610 +-- To enable: `local colors = require 'term.colors'` +local colors = setmetatable({}, {__index = function() return function(s) return s end end}) return function(options) local busted = require 'busted' diff --git a/test/functional/api/server_notifications_spec.lua b/test/functional/api/server_notifications_spec.lua index 279ede81f7..6367cc5caa 100644 --- a/test/functional/api/server_notifications_spec.lua +++ b/test/functional/api/server_notifications_spec.lua @@ -78,10 +78,10 @@ describe('notify', function() end) it('cancels stale events on channel close', function() - if isCI() then - pending('Sporadic hangs on CI (c.f., #14083). Skip until it is fixed.') - return - end + if isCI() then + pending('hangs on CI #14083 #15251') + return + end if helpers.pending_win32(pending) then return end local catchan = eval("jobstart(['cat'], {'rpc': v:true})") local catpath = eval('exepath("cat")') diff --git a/test/functional/lua/mpack_spec.lua b/test/functional/lua/mpack_spec.lua new file mode 100644 index 0000000000..ef693f01f3 --- /dev/null +++ b/test/functional/lua/mpack_spec.lua @@ -0,0 +1,23 @@ +-- Test suite for testing interactions with API bindings +local helpers = require('test.functional.helpers')(after_each) + +local clear = helpers.clear +local eq = helpers.eq +local exec_lua = helpers.exec_lua + +describe('lua vim.mpack', function() + before_each(clear) + it('can pack vim.NIL', function() + eq({true, true, true, true}, exec_lua [[ + local var = vim.mpack.unpack(vim.mpack.pack({33, vim.NIL, 77})) + return {var[1]==33, var[2]==vim.NIL, var[3]==77, var[4]==nil} + ]]) + end) + + it('can pack vim.empty_dict()', function() + eq({{{}, "foo", {}}, true, false}, exec_lua [[ + local var = vim.mpack.unpack(vim.mpack.pack({{}, "foo", vim.empty_dict()})) + return {var, vim.tbl_islist(var[1]), vim.tbl_islist(var[3])} + ]]) + end) +end) diff --git a/test/functional/lua/vim_spec.lua b/test/functional/lua/vim_spec.lua index 0ea914880f..2bedbd1453 100644 --- a/test/functional/lua/vim_spec.lua +++ b/test/functional/lua/vim_spec.lua @@ -6,6 +6,7 @@ local funcs = helpers.funcs local meths = helpers.meths local dedent = helpers.dedent local command = helpers.command +local insert = helpers.insert local clear = helpers.clear local eq = helpers.eq local ok = helpers.ok @@ -1881,7 +1882,7 @@ describe('lua stdlib', function() end) it('vim.region', function() - helpers.insert(helpers.dedent( [[ + insert(helpers.dedent( [[ text tααt tααt text text tαxt txtα tex text tαxt tαxt @@ -1889,65 +1890,67 @@ describe('lua stdlib', function() eq({5,15}, exec_lua[[ return vim.region(0,{1,5},{1,14},'v',true)[1] ]]) end) - describe('vim.execute_on_keystroke', function() - it('should keep track of keystrokes', function() - helpers.insert([[hello world ]]) + describe('vim.on_key', function() + it('tracks keystrokes', function() + insert([[hello world ]]) exec_lua [[ - KeysPressed = {} + keys = {} - vim.register_keystroke_callback(function(buf) + vim.on_key(function(buf) if buf:byte() == 27 then buf = "<ESC>" end - table.insert(KeysPressed, buf) + table.insert(keys, buf) end) ]] - helpers.insert([[next 🤦 lines å ]]) + insert([[next 🤦 lines å ]]) -- It has escape in the keys pressed - eq('inext 🤦 lines å <ESC>', exec_lua [[return table.concat(KeysPressed, '')]]) + eq('inext 🤦 lines å <ESC>', exec_lua [[return table.concat(keys, '')]]) end) - it('should allow removing trackers.', function() - helpers.insert([[hello world]]) + it('allows removing on_key listeners', function() + insert([[hello world]]) exec_lua [[ - KeysPressed = {} + keys = {} - return vim.register_keystroke_callback(function(buf) + return vim.on_key(function(buf) if buf:byte() == 27 then buf = "<ESC>" end - table.insert(KeysPressed, buf) + table.insert(keys, buf) end, vim.api.nvim_create_namespace("logger")) ]] - helpers.insert([[next lines]]) + insert([[next lines]]) - exec_lua("vim.register_keystroke_callback(nil, vim.api.nvim_create_namespace('logger'))") + eq(1, exec_lua('return vim.on_key()')) + exec_lua("vim.on_key(nil, vim.api.nvim_create_namespace('logger'))") + eq(0, exec_lua('return vim.on_key()')) - helpers.insert([[more lines]]) + insert([[more lines]]) -- It has escape in the keys pressed - eq('inext lines<ESC>', exec_lua [[return table.concat(KeysPressed, '')]]) + eq('inext lines<ESC>', exec_lua [[return table.concat(keys, '')]]) end) - it('should not call functions that error again.', function() - helpers.insert([[hello world]]) + it('skips any function that caused an error', function() + insert([[hello world]]) exec_lua [[ - KeysPressed = {} + keys = {} - return vim.register_keystroke_callback(function(buf) + return vim.on_key(function(buf) if buf:byte() == 27 then buf = "<ESC>" end - table.insert(KeysPressed, buf) + table.insert(keys, buf) if buf == 'l' then error("Dumb Error") @@ -1955,35 +1958,30 @@ describe('lua stdlib', function() end) ]] - helpers.insert([[next lines]]) - helpers.insert([[more lines]]) + insert([[next lines]]) + insert([[more lines]]) -- Only the first letter gets added. After that we remove the callback - eq('inext l', exec_lua [[ return table.concat(KeysPressed, '') ]]) + eq('inext l', exec_lua [[ return table.concat(keys, '') ]]) end) - it('should process mapped keys, not unmapped keys', function() + it('processes mapped keys, not unmapped keys', function() exec_lua [[ - KeysPressed = {} + keys = {} vim.cmd("inoremap hello world") - vim.register_keystroke_callback(function(buf) + vim.on_key(function(buf) if buf:byte() == 27 then buf = "<ESC>" end - table.insert(KeysPressed, buf) + table.insert(keys, buf) end) ]] + insert("hello") - helpers.insert("hello") - - local next_status = exec_lua [[ - return table.concat(KeysPressed, '') - ]] - - eq("iworld<ESC>", next_status) + eq('iworld<ESC>', exec_lua[[return table.concat(keys, '')]]) end) end) diff --git a/test/functional/ui/multigrid_spec.lua b/test/functional/ui/multigrid_spec.lua index 719e2ee82a..03cb43a7d1 100644 --- a/test/functional/ui/multigrid_spec.lua +++ b/test/functional/ui/multigrid_spec.lua @@ -2220,4 +2220,92 @@ describe('ext_multigrid', function() [4] = {win = {id = 1001}, topline = 4, botline = 8, curline = 4, curcol = 38}, }} end) + + it('does not crash when dragging mouse across grid boundary', function() + screen:try_resize(48, 8) + screen:expect{grid=[[ + ## grid 1 + [2:------------------------------------------------]| + [2:------------------------------------------------]| + [2:------------------------------------------------]| + [2:------------------------------------------------]| + [2:------------------------------------------------]| + [2:------------------------------------------------]| + {11:[No Name] }| + [3:------------------------------------------------]| + ## grid 2 + ^ | + {1:~ }| + {1:~ }| + {1:~ }| + {1:~ }| + {1:~ }| + ## grid 3 + | + ]], win_viewport={ + [2] = {win = { id = 1000 }, topline = 0, botline = 2, curline = 0, curcol = 0} + }} + insert([[ + Lorem ipsum dolor sit amet, consectetur + adipisicing elit, sed do eiusmod tempor + incididunt ut labore et dolore magna aliqua. + Ut enim ad minim veniam, quis nostrud + exercitation ullamco laboris nisi ut aliquip ex + ea commodo consequat. Duis aute irure dolor in + reprehenderit in voluptate velit esse cillum + dolore eu fugiat nulla pariatur. Excepteur sint + occaecat cupidatat non proident, sunt in culpa + qui officia deserunt mollit anim id est + laborum.]]) + + screen:expect{grid=[[ + ## grid 1 + [2:------------------------------------------------]| + [2:------------------------------------------------]| + [2:------------------------------------------------]| + [2:------------------------------------------------]| + [2:------------------------------------------------]| + [2:------------------------------------------------]| + {11:[No Name] [+] }| + [3:------------------------------------------------]| + ## grid 2 + ea commodo consequat. Duis aute irure dolor in | + reprehenderit in voluptate velit esse cillum | + dolore eu fugiat nulla pariatur. Excepteur sint | + occaecat cupidatat non proident, sunt in culpa | + qui officia deserunt mollit anim id est | + laborum^. | + ## grid 3 + | + ]], win_viewport={ + [2] = {win = {id = 1000}, topline = 5, botline = 11, curline = 10, curcol = 7}, + }} + + meths.input_mouse('left', 'press', '', 1,5, 1) + poke_eventloop() + meths.input_mouse('left', 'drag', '', 1, 6, 1) + + screen:expect{grid=[[ + ## grid 1 + [2:------------------------------------------------]| + [2:------------------------------------------------]| + [2:------------------------------------------------]| + [2:------------------------------------------------]| + [2:------------------------------------------------]| + [2:------------------------------------------------]| + {11:[No Name] [+] }| + [3:------------------------------------------------]| + ## grid 2 + reprehenderit in voluptate velit esse cillum | + dolore eu fugiat nulla pariatur. Excepteur sint | + occaecat cupidatat non proident, sunt in culpa | + qui officia deserunt mollit anim id est | + l^aborum. | + {1:~ }| + ## grid 3 + {7:-- VISUAL --} | + ]], win_viewport={ + [2] = {win = {id = 1000}, topline = 6, botline = 12, curline = 10, curcol = 1}, + }} + end) end) |