From 2cdea852e8934beb89012f2127f333e4dd8aada8 Mon Sep 17 00:00:00 2001
From: Jongwook Choi <wookayin@gmail.com>
Date: Thu, 11 Jan 2024 12:24:44 -0500
Subject: docs: auto-generate docs for `vim.lpeg` and `vim.re`

- Add section `VIM.LPEG` and `VIM.RE` to docs/lua.txt.

- Add `_meta/re.lua` which adds luadoc and type annotations, for the
  vendored `vim.re` package.

- Fix minor style issues on `_meta/lpeg.lua` luadoc for better vimdocs
  generation.

- Fix a bug on `gen_vimdoc` where non-helptags in verbatim code blocks
  were parsed as helptags, affecting code examples on `vim.lpeg.Cf`,
  etc.

- Also move the `vim.regex` section below so that it can be located
  closer to `vim.lpeg` and `vim.re`.
---
 runtime/lua/vim/_meta/re.lua | 57 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)
 create mode 100644 runtime/lua/vim/_meta/re.lua

(limited to 'runtime/lua/vim/_meta/re.lua')

diff --git a/runtime/lua/vim/_meta/re.lua b/runtime/lua/vim/_meta/re.lua
new file mode 100644
index 0000000000..4f254b19a0
--- /dev/null
+++ b/runtime/lua/vim/_meta/re.lua
@@ -0,0 +1,57 @@
+--- @meta
+error('Cannot require a meta file')
+
+-- Documentations and Lua types for vim.re (vendored re.lua, lpeg-1.1.0)
+-- https://www.inf.puc-rio.br/~roberto/lpeg/re.html
+--
+-- Copyright © 2007-2023 Lua.org, PUC-Rio.
+-- See 'lpeg.html' for license
+
+--- @defgroup vim.re
+---<pre>help
+---The `vim.re` module provides a conventional regex-like syntax for pattern usage
+---within LPeg |vim.lpeg|.
+---
+---See https://www.inf.puc-rio.br/~roberto/lpeg/re.html for the original
+---documentation including regex syntax and more concrete examples.
+---
+---</pre>
+
+--- Compiles the given {string} and returns an equivalent LPeg pattern. The given string may define
+--- either an expression or a grammar. The optional {defs} table provides extra Lua values to be used
+--- by the pattern.
+--- @param string string
+--- @param defs? table
+--- @return vim.lpeg.Pattern
+function vim.re.compile(string, defs) end
+
+--- Searches the given {pattern} in the given {subject}. If it finds a match, returns the index
+--- where this occurrence starts and the index where it ends. Otherwise, returns nil.
+---
+--- An optional numeric argument {init} makes the search starts at that position in the subject
+--- string. As usual in Lua libraries, a negative value counts from the end.
+--- @param subject string
+--- @param pattern vim.lpeg.Pattern|string
+--- @param init? integer
+--- @return integer|nil the index where the occurrence starts, nil if no match
+--- @return integer|nil the index where the occurrence ends, nil if no match
+function vim.re.find(subject, pattern, init) end
+
+--- Does a global substitution, replacing all occurrences of {pattern} in the given {subject} by
+--- {replacement}.
+--- @param subject string
+--- @param pattern vim.lpeg.Pattern|string
+--- @param replacement string
+--- @return string
+function vim.re.gsub(subject, pattern, replacement) end
+
+--- Matches the given {pattern} against the given {subject}, returning all captures.
+--- @param subject string
+--- @param pattern vim.lpeg.Pattern|string
+--- @param init? integer
+--- @return integer|vim.lpeg.Capture|nil
+--- @see vim.lpeg.match()
+function vim.re.match(subject, pattern, init) end
+
+--- Updates the pre-defined character classes to the current locale.
+function vim.re.updatelocale() end
-- 
cgit 


From 9beb40a4db5613601fc1a4b828a44e5977eca046 Mon Sep 17 00:00:00 2001
From: Lewis Russell <lewis6991@gmail.com>
Date: Thu, 15 Feb 2024 17:16:04 +0000
Subject: feat(docs): replace lua2dox.lua

Problem:

The documentation flow (`gen_vimdoc.py`) has several issues:
- it's not very versatile
- depends on doxygen
- doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C.
- The intermediate XML files and filters makes it too much like a rube goldberg machine.

Solution:

Re-implement the flow using Lua, LPEG and treesitter.

- `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic.
- `lua2dox.lua` is gone!
- No more XML files.
- Doxygen is now longer used and instead we now use:
  - LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`).
  - LPEG for C parsing (see `scripts/cdoc_parser.lua`)
  - Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`).
  - Treesitter for Markdown parsing (see `scripts/text_utils.lua`).
- The generated `runtime/doc/*.mpack` files have been removed.
   - `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly.
- Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).
---
 runtime/lua/vim/_meta/re.lua | 13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)

(limited to 'runtime/lua/vim/_meta/re.lua')

diff --git a/runtime/lua/vim/_meta/re.lua b/runtime/lua/vim/_meta/re.lua
index 4f254b19a0..14c94c7824 100644
--- a/runtime/lua/vim/_meta/re.lua
+++ b/runtime/lua/vim/_meta/re.lua
@@ -7,15 +7,12 @@ error('Cannot require a meta file')
 -- Copyright © 2007-2023 Lua.org, PUC-Rio.
 -- See 'lpeg.html' for license
 
---- @defgroup vim.re
----<pre>help
----The `vim.re` module provides a conventional regex-like syntax for pattern usage
----within LPeg |vim.lpeg|.
+--- @brief
+--- The `vim.re` module provides a conventional regex-like syntax for pattern usage
+--- within LPeg |vim.lpeg|.
 ---
----See https://www.inf.puc-rio.br/~roberto/lpeg/re.html for the original
----documentation including regex syntax and more concrete examples.
----
----</pre>
+--- See https://www.inf.puc-rio.br/~roberto/lpeg/re.html for the original
+--- documentation including regex syntax and more concrete examples.
 
 --- Compiles the given {string} and returns an equivalent LPeg pattern. The given string may define
 --- either an expression or a grammar. The optional {defs} table provides extra Lua values to be used
-- 
cgit