feat(docs): replace lua2dox.lua

Problem:

The documentation flow (`gen_vimdoc.py`) has several issues:
- it's not very versatile
- depends on doxygen
- doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C.
- The intermediate XML files and filters makes it too much like a rube goldberg machine.

Solution:

Re-implement the flow using Lua, LPEG and treesitter.

- `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic.
- `lua2dox.lua` is gone!
- No more XML files.
- Doxygen is now longer used and instead we now use:
  - LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`).
  - LPEG for C parsing (see `scripts/cdoc_parser.lua`)
  - Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`).
  - Treesitter for Markdown parsing (see `scripts/text_utils.lua`).
- The generated `runtime/doc/*.mpack` files have been removed.
   - `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly.
- Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).

1 files changed, 5 insertions, 8 deletions

diff --git a/runtime/lua/vim/loader.lua b/runtime/lua/vim/loader.lua
index ec99e417c2..5f3da55544 100644
--- a/runtime/lua/vim/loader.lua
+++ b/runtime/lua/vim/loader.lua
@@ -190,7 +190,6 @@ function Loader.loader_lib(modname)
   local sysname = uv.os_uname().sysname:lower() or ''
   local is_win = sysname:find('win', 1, true) and not sysname:find('darwin', 1, true)
   local ret = M.find(modname, { patterns = is_win and { '.dll' } or { '.so' } })[1]
-  ---@type function?, string?
   if ret then
     -- Making function name in Lua 5.1 (see src/loadlib.c:mkfuncname) is
     -- a) strip prefix up to and including the first dash, if any
@@ -208,15 +207,13 @@ end
 --- `loadfile` using the cache
 --- Note this has the mode and env arguments which is supported by LuaJIT and is 5.1 compatible.
 ---@param filename? string
----@param mode? "b"|"t"|"bt"
+---@param _mode? "b"|"t"|"bt"
 ---@param env? table
 ---@return function?, string?  error_message
 ---@private
--- luacheck: ignore 312
-function Loader.loadfile(filename, mode, env)
+function Loader.loadfile(filename, _mode, env)
   -- ignore mode, since we byte-compile the Lua source files
-  mode = nil
-  return Loader.load(normalize(filename), { mode = mode, env = env })
+  return Loader.load(normalize(filename), { env = env })
 end
 
 --- Checks whether two cache hashes are the same based on:
@@ -273,14 +270,14 @@ end
 
 --- Finds Lua modules for the given module name.
 ---@param modname string Module name, or `"*"` to find the top-level modules instead
----@param opts? ModuleFindOpts (table|nil) Options for finding a module:
+---@param opts? ModuleFindOpts (table) Options for finding a module:
 ---    - rtp: (boolean) Search for modname in the runtime path (defaults to `true`)
 ---    - paths: (string[]) Extra paths to search for modname (defaults to `{}`)
 ---    - patterns: (string[]) List of patterns to use when searching for modules.
 ---                A pattern is a string added to the basename of the Lua module being searched.
 ---                (defaults to `{"/init.lua", ".lua"}`)
 ---    - all: (boolean) Return all matches instead of just the first one (defaults to `false`)
----@return ModuleInfo[] (list) A list of results with the following properties:
+---@return ModuleInfo[] (table) A list of results with the following properties:
 ---    - modpath: (string) the path to the module
 ---    - modname: (string) the name of the module
 ---    - stat: (table|nil) the fs_stat of the module path. Won't be returned for `modname="*"`