Merge remote-tracking branch 'upstream/master' into userreguserreg

1 files changed, 99 insertions, 94 deletions

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index ce0f2ace05..4f9c0d720d 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -8,9 +8,7 @@ If you want to help but don't know where to start, here are some
 low-risk/isolated tasks:
 
 - Try a [complexity:low] issue.
-- Fix bugs found by [Clang](#clang-scan-build), [PVS](#pvs-studio) or
-  [Coverity](#coverity).
-- [Improve documentation](#documenting)
+- Fix bugs found by [Clang](#clang-scan-build) or [Coverity](#coverity).
 - [Merge a Vim patch] (requires strong familiarity with Vim)
   - NOTE: read the above link before sending improvements to "runtime files" (anything in `runtime/`).
     - Vimscript and documentation files are (mostly) maintained by [Vim](https://github.com/vim/vim), not Nvim.
@@ -33,15 +31,20 @@ Reporting problems
 Developer guidelines
 --------------------
 
-- Read [:help dev](https://neovim.io/doc/user/develop.html#dev) if you are working on Nvim core.
+- Read [:help dev](https://neovim.io/doc/user/develop.html#dev) and [:help dev-doc][dev-doc-guide] if you are working on Nvim core.
 - Read [:help dev-ui](https://neovim.io/doc/user/develop.html#dev-ui) if you are developing a UI.
 - Read [:help dev-api-client](https://neovim.io/doc/user/develop.html#dev-api-client) if you are developing an API client.
 - Install `ninja` for faster builds of Nvim.
-  ```
+  ```bash
   sudo apt-get install ninja-build
   make distclean
   make  # Nvim build system uses ninja automatically, if available.
   ```
+- Install `ccache` for faster rebuilds of Nvim. Nvim will use it automatically
+  if it's found. To disable caching use:
+  ```bash
+  CCACHE_DISABLE=true make
+  ```
 
 Pull requests (PRs)
 ---------------------
@@ -50,12 +53,8 @@ Pull requests (PRs)
 - Your PR must include [test coverage][run-tests].
 - Avoid cosmetic changes to unrelated files in the same commit.
 - Use a [feature branch][git-feature-branch] instead of the master branch.
-- Use a _rebase workflow_ for small PRs.
+- Use a _rebase workflow_ for all PRs.
   - After addressing review comments, it's fine to force-push.
-- Use a _merge workflow_ (as opposed to "rebase") for big, high-risk PRs.
-  - Merge `master` into your PR when there are conflicts or when master
-    introduces breaking changes.
-  - Do not edit commits that come before the merge commit.
 
 ### Merging to master
 
@@ -93,7 +92,7 @@ the VCS/git logs more valuable. The general structure of a commit message is:
 ```
 
 - Prefix the commit subject with one of these [_types_](https://github.com/commitizen/conventional-commit-types/blob/master/index.json):
-    - `build`, `ci`, `docs`, `feat`, `fix`, `perf`, `refactor`, `revert`, `test`, `vim-patch`, `dist`
+    - `build`, `ci`, `docs`, `feat`, `fix`, `perf`, `refactor`, `revert`, `test`, `vim-patch`
     - You can **ignore this for "fixup" commits** or any commits you expect to be squashed.
 - Append optional scope to _type_ such as `(lsp)`, `(treesitter)`, `(float)`, …
 - _Description_ shouldn't start with a capital letter or end in a period.
@@ -110,6 +109,11 @@ the VCS/git logs more valuable. The general structure of a commit message is:
        BREAKING CHANGE: refactor to use Python 3 features since Python 2 is no longer supported.
        ```
 
+### News
+
+High level release notes are maintained in [news.txt](runtime/doc/news.txt). A PR is not required to add a news item
+but is generally recommended.
+
 ### Automated builds (CI)
 
 Each pull request must pass the automated builds on [Cirrus CI] and [GitHub Actions].
@@ -119,13 +123,13 @@ Each pull request must pass the automated builds on [Cirrus CI] and [GitHub Acti
 - If any tests fail, the build will fail. See [test/README.md#running-tests][run-tests] to run tests locally.
 - CI runs [ASan] and other analyzers.
     - To run valgrind locally: `VALGRIND=1 make test`
-    - To run Clang ASan/UBSan locally: `CC=clang make CMAKE_FLAGS="-DCLANG_ASAN_UBSAN=ON"`
-- The [lint](#lint) build checks modified lines _and their immediate
-  neighbors_, to encourage incrementally updating the legacy style to meet our
-  [style](#style). (See [#3174][3174] for background.)
+    - To run ASan/UBSan locally: `CC=clang make CMAKE_FLAGS="-DENABLE_ASAN_UBSAN=ON"`.
+      Note that MSVC requires Release or RelWithDebInfo build type to work properly.
+- The [lint](#lint) build checks that the code is formatted correctly and
+  passes various linter checks.
 - CI for FreeBSD runs on [Cirrus CI].
 - To see CI results faster in your PR, you can temporarily set `TEST_FILE` in
-  [ci.yml](https://github.com/neovim/neovim/blob/e35b9020b16985eee26e942f9a3f6b045bc3809b/.github/workflows/ci.yml#L205).
+  [test.yml](https://github.com/neovim/neovim/blob/e35b9020b16985eee26e942f9a3f6b045bc3809b/.github/workflows/test.yml#L29).
 
 ### Clang scan-build
 
@@ -133,30 +137,15 @@ View the [Clang report] to see potential bugs found by the Clang
 [scan-build](https://clang-analyzer.llvm.org/scan-build.html) analyzer.
 
 - Search the Neovim commit history to find examples:
-  ```
+  ```bash
   git log --oneline --no-merges --grep clang
   ```
 - To verify a fix locally, run `scan-build` like this:
-  ```
+  ```bash
   rm -rf build/
   scan-build --use-analyzer=/usr/bin/clang make
   ```
 
-### PVS-Studio
-
-View the [PVS report](https://neovim.io/doc/reports/pvs/PVS-studio.html.d/) to
-see potential bugs found by [PVS Studio](https://www.viva64.com/en/pvs-studio/).
-
-- Use this format for commit messages (where `{id}` is the PVS warning-id)):
-  ```
-  fix(PVS/V{id}): {description}
-  ```
-- Search the Neovim commit history to find examples:
-  ```
-  git log --oneline --no-merges --grep PVS
-  ```
-- Try `./scripts/pvscheck.sh` to run PVS locally.
-
 ### Coverity
 
 [Coverity](https://scan.coverity.com/projects/neovim-neovim) runs against the
@@ -168,21 +157,21 @@ master build. To view the defects, just request access; you will be approved.
   fix(coverity/{id}): {description}
   ```
 - Search the Neovim commit history to find examples:
-  ```
+  ```bash
   git log --oneline --no-merges --grep coverity
   ```
 
-### Clang sanitizers (ASAN and UBSAN)
+### Sanitizers (ASAN and UBSAN)
 
   ASAN/UBSAN can be used to detect memory errors and other common forms of undefined behavior at runtime in debug builds.
 
 - To build Neovim with sanitizers enabled, use
   ```
-  rm -rf build && CMAKE_EXTRA_FLAGS="-DCMAKE_C_COMPILER=clang -DCLANG_ASAN_UBSAN=1" make
+  rm -rf build && CMAKE_EXTRA_FLAGS="-DCMAKE_C_COMPILER=clang -DENABLE_ASAN_UBSAN=1" make
   ```
 - When running Neovim, use
   ```
-  UBSAN_OPTIONS=print_stacktrace=1 ASAN_OPTIONS=log_path=/tmp/nvim_asan nvim args...
+  ASAN_OPTIONS=log_path=/tmp/nvim_asan nvim args...
   ```
 - If Neovim exits unexpectedly, check `/tmp/nvim_asan.{PID}` (or your preferred `log_path`) for log files with error messages.
 
@@ -194,33 +183,27 @@ Coding
 
 You can run the linter locally by:
 
-    make lint
-
-The lint step downloads the [master error list] and excludes them, so only lint
-errors related to the local changes are reported.
-
-You can lint a single file (but this will _not_ exclude legacy errors):
-
-    ./src/clint.py src/nvim/ops.c
+```bash
+make lint
+```
 
 ### Style
 
 - You can format files by using:
-  ```
-    make format
+  ```bash
+  make format  # or formatc, formatlua
   ```
   This will format changed Lua and C files with all appropriate flags set.
 - Style rules are (mostly) defined by `src/uncrustify.cfg` which tries to match
   the [style-guide]. To use the Nvim `gq` command with `uncrustify`:
-  ```
+  ```vim
   if !empty(findfile('src/uncrustify.cfg', ';'))
     setlocal formatprg=uncrustify\ -q\ -l\ C\ -c\ src/uncrustify.cfg\ --no-backup
   endif
   ```
-  The required version of `uncrustify` is specified in `uncrustify.cfg`.
 - There is also `.clang-format` which has drifted from the [style-guide], but
   is available for reference. To use the Nvim `gq` command with `clang-format`:
-  ```
+  ```vim
   if !empty(findfile('.clang-format', ';'))
     setlocal formatprg=clang-format\ -style=file
   endif
@@ -228,18 +211,14 @@ You can lint a single file (but this will _not_ exclude legacy errors):
 
 ### Navigate
 
-- Set `blame.ignoreRevsFile` to ignore [noise commits](https://github.com/neovim/neovim/commit/2d240024acbd68c2d3f82bc72cb12b1a4928c6bf) in git blame:
-  ```
+- Set `blame.ignoreRevsFile` to ignore [noisy commits](https://github.com/neovim/neovim/commit/2d240024acbd68c2d3f82bc72cb12b1a4928c6bf) in git blame:
+  ```bash
   git config blame.ignoreRevsFile .git-blame-ignore-revs
   ```
 
 - Recommendation is to use **[clangd]**.
   Can use the maintained config in [nvim-lspconfig/clangd].
 - Explore the source code [on the web](https://sourcegraph.com/github.com/neovim/neovim).
-- If using [lua-language-server], symlink `contrib/luarc.json` into the
-  project root:
-
-      $ ln -s contrib/luarc.json .luarc.json
 
 ### Includes
 
@@ -247,62 +226,92 @@ For managing includes in C files, use [include-what-you-use].
 
 - [Install include-what-you-use][include-what-you-use-install]
 - To see which includes needs fixing use the cmake preset `iwyu`:
-  ```
-  cmake --workflow --preset iwyu
+  ```bash
+  cmake --preset iwyu
+  cmake --build build
   ```
 - There's also a make target that automatically fixes the suggestions from
   IWYU:
-  ```
+  ```bash
   make iwyu
   ```
 
 See [#549][549] for more details.
 
-Documenting
------------
+### Lua runtime files
 
-Many parts of the `:help` documentation are autogenerated from C or Lua docstrings using the `./scripts/gen_vimdoc.py` script.
-You can filter the regeneration based on the target (api, lua, or lsp), or the file you changed, that need a doc refresh using `./scripts/gen_vimdoc.py -t <target>`.
+Most of the Lua core [`runtime/`](./runtime) modules are precompiled to
+bytecode, so changes to those files won't get used unless you rebuild Nvim or
+by passing `--luamod-dev` and `$VIMRUNTIME`. For example, try adding a function
+to `runtime/lua/vim/_editor.lua` then:
 
-## Lua docstrings
+```bash
+VIMRUNTIME=./runtime ./build/bin/nvim --luamod-dev
+```
 
-Lua documentation uses a subset of [EmmyLua] annotations. A rough outline of a function documentation is
+Documentation
+-------------
 
-```lua
---- {Brief}
----
---- {Long explanation}
----
----@param arg1 type {description}
----@param arg2 type {description}
-{...}
----
----@return type {description}
-```
+Read [:help dev-doc][dev-doc-guide] to understand the expected documentation style and conventions.
 
-If possible, always add type information (`table`, `string`, `number`, ...). Multiple valid types are separated by a bar (`string|table`). Indicate optional parameters via `type|nil`.
+### Generating :help
 
-If a function in your Lua module should not be documented (e.g. internal function or local function), you should set the doc comment to:
+Many `:help` docs are autogenerated from (C or Lua) docstrings. To generate the documentation run:
 
-```
----@private
+```bash
+make doc
 ```
 
-Mark functions that are deprecated as 
-```
----@deprecated
-```
+If you need to modify or debug the documentation flow, these are the main files:
+- `./scripts/gen_vimdoc.py`:
+  Main doc generator. Drives doxygen to generate xml files, and scrapes those
+  xml files to render vimdoc files.
+- `./scripts/lua2dox.lua`:
+  Used by `gen_vimdoc.py` to transform Lua files into a format compatible with doxygen.
+- `./scripts/gen_eval_files.lua`:
+  Generates documentation and Lua type files from metadata files:
+  ```
+  runtime/lua/vim/*     =>  runtime/doc/lua.txt
+  runtime/lua/vim/*     =>  runtime/doc/lua.txt
+  runtime/lua/vim/lsp/  =>  runtime/doc/lsp.txt
+  src/nvim/api/*        =>  runtime/doc/api.txt
+  src/nvim/eval.lua     =>  runtime/doc/builtin.txt
+  src/nvim/options.lua  =>  runtime/doc/options.txt
+  ```
+
+### Lua docstrings
+
+Use [LuaLS] annotations in Lua docstrings to annotate parameter types, return
+types, etc. See [:help dev-doc-lua][dev-doc-lua].
+
+- The template for function documentation is:
+  ```lua
+  --- {Brief}
+  ---
+  --- {Long explanation}
+  ---
+  --- @param arg1 type {description}
+  --- @param arg2 type {description}
+  --- ...
+  ---
+  --- @return type {description}
+  ```
+- If possible, add type information (`table`, `string`, `number`, ...). Multiple valid types are separated by a bar (`string|table`). Indicate optional parameters via `type|nil`.
+- If a function in your Lua module should _not_ be documented, add `@nodoc`.
+- If the function is internal or otherwise non-public add `@private`.
+      - Private functions usually should be underscore-prefixed (named "_foo", not "foo").
+- Mark deprecated functions with `@deprecated`.
 
 Reviewing
 ---------
 
-To help review pull requests, start with [this checklist][review-checklist].
-
 Reviewing can be done on GitHub, but you may find it easier to do locally.
 Using [GitHub CLI][gh], you can create a new branch with the contents of a pull
 request, e.g. [#1820][1820]:
 
-    gh pr checkout https://github.com/neovim/neovim/pull/1820
+```bash
+gh pr checkout https://github.com/neovim/neovim/pull/1820
+```
 
 Use [`git log -p master..FETCH_HEAD`][git-history-filtering] to list all
 commits in the feature branch which aren't in the `master` branch; `-p`
@@ -320,25 +329,21 @@ as context, use the `-W` argument as well.
 [Merge a Vim patch]: https://github.com/neovim/neovim/wiki/Merging-patches-from-upstream-Vim
 [complexity:low]: https://github.com/neovim/neovim/issues?q=is%3Aopen+is%3Aissue+label%3Acomplexity%3Alow
 [conventional_commits]: https://www.conventionalcommits.org
-[EmmyLua]: https://github.com/sumneko/lua-language-server/wiki/Annotations
+[dev-doc-guide]: https://neovim.io/doc/user/develop.html#dev-doc
+[dev-doc-lua]: https://neovim.io/doc/user/develop.html#dev-lua-doc
+[LuaLS]: https://github.com/LuaLS/lua-language-server/wiki/Annotations
 [gcc-warnings]: https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html
 [gh]: https://cli.github.com/
 [git-bisect]: http://git-scm.com/book/en/v2/Git-Tools-Debugging-with-Git
 [git-feature-branch]: https://www.atlassian.com/git/tutorials/comparing-workflows
 [git-history-filtering]: https://www.atlassian.com/git/tutorials/git-log/filtering-the-commit-history
-[git-history-rewriting]: http://git-scm.com/book/en/v2/Git-Tools-Rewriting-History
-[git-rebasing]: http://git-scm.com/book/en/v2/Git-Branching-Rebasing
 [github-issues]: https://github.com/neovim/neovim/issues
 [include-what-you-use-install]: https://github.com/include-what-you-use/include-what-you-use#how-to-install
 [include-what-you-use]: https://github.com/include-what-you-use/include-what-you-use#using-with-cmake
 [lua-language-server]: https://github.com/sumneko/lua-language-server/
-[master error list]: https://raw.githubusercontent.com/neovim/doc/gh-pages/reports/clint/errors.json
 [nvim-lspconfig/clangd]: https://github.com/neovim/nvim-lspconfig/blob/master/doc/server_configurations.md#clangd
 [pr-draft]: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request
 [pr-ready]: https://docs.github.com/en/github/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-stage-of-a-pull-request
-[review-checklist]: https://github.com/neovim/neovim/wiki/Code-review-checklist
 [run-tests]: https://github.com/neovim/neovim/blob/master/test/README.md#running-tests
 [style-guide]: https://neovim.io/doc/user/dev_style.html#dev-style
-[uncrustify]: http://uncrustify.sourceforge.net/
-[wiki-contribute-help]: https://github.com/neovim/neovim/wiki/contribute-%3Ahelp
 [wiki-faq]: https://github.com/neovim/neovim/wiki/FAQ