Problem:
The magic globals `it`, `describe`, etc., are more trouble than they are
worth.

- Hooking into `after_each` requires `getfenv()` hacks.
- They confuse luals/emmylua, because the top-level `.luarc.json` isn't
  merged with `test/.luarc.json` (apparently a luals limitation?)
- They totally defeat discoverability because the user just has to
  "know" about the various magic symbols.

So they harm DX, which means they serve no purpose at all.

Solution:
- Expose the test API from `testutil`, so tests can call `t.it()`,
  `t.describe()`, etc., in the conventional way.
- Drop `getfenv()` hacks.
- Drop the `setfenv()` injection in `load_chunk`.
- Drop `test/_meta.lua`.

- Move all generator Lua scripts to the `src/gen/`
- Add a `.luarc.json` to `src/gen/`
- Add a `preload.lua` to `src/gen/`
  - Add `src` to `package.path` so it aligns with `.luarc.json'
- Fix all `require` statements in `src/gen/` so they are consistent:
    - `require('scripts.foo')` -> `require('gen.foo')`
    - `require('src.nvim.options')` -> `require('nvim.options')`
    - `require('api.dispatch_deprecated')` -> `require('nvim.api.dispatch_deprecated')`

An implication of this current approach is that `NVIM_API_LEVEL` should be
bumped when a new Lua function is added.

TODO(future): add a lint check which requires `@since` on all new functions.

ref #25416

Specifically, functions that are run in the context of the test runner
are put in module `test/testutil.lua` while the functions that are run
in the context of the test session are put in
`test/functional/testnvim.lua`.

Closes https://github.com/neovim/neovim/issues/27004.

Work on https://github.com/neovim/neovim/issues/27004.

- Tags are now created with `[tag]()`
- References are now created with `[tag]`
- Code spans are no longer wrapped

- Added `@inlinedoc` so single use Lua types can be inlined into the
  functions docs. E.g.

  ```lua
  --- @class myopts
  --- @inlinedoc
  ---
  --- Documentation for some field
  --- @field somefield integer

  --- @param opts myOpts
  function foo(opts)
  end
  ```

  Will be rendered as

  ```
  foo(opts)

    Parameters:
      - {opts} (table) Object with the fields:
               - somefield (integer) Documentation
                 for some field
  ```

- Marked many classes with with `@nodoc` or `(private)`.
  We can eventually introduce these when we want to.

Problem:

The documentation flow (`gen_vimdoc.py`) has several issues:
- it's not very versatile
- depends on doxygen
- doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C.
- The intermediate XML files and filters makes it too much like a rube goldberg machine.

Solution:

Re-implement the flow using Lua, LPEG and treesitter.

- `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic.
- `lua2dox.lua` is gone!
- No more XML files.
- Doxygen is now longer used and instead we now use:
  - LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`).
  - LPEG for C parsing (see `scripts/cdoc_parser.lua`)
  - Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`).
  - Treesitter for Markdown parsing (see `scripts/text_utils.lua`).
- The generated `runtime/doc/*.mpack` files have been removed.
   - `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly.
- Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).