aboutsummaryrefslogtreecommitdiff
path: root/cmake/RunTests.cmake
Commit message (Collapse)AuthorAge
* build(test): dead manifest cleanup in RunTests.cmake #41514Willaaaaaaa2026-09-11
| | | NVIM_RPLUGIN_MANIFEST was removed in f034e5f66b74cafb027dce157eef36e8766e0e7e
* refactor(test): unify XDG path handling (cmake vs zig)bfredl2026-07-06
| | | | fixes #40381
* test: run Lua harness with nvim -lLewis Russell2026-05-13
| | | | | | | | | | | | | | | | | | Problem: The Lua test harness still ran through standalone -ll mode, so tests depended on the low-level Lua path instead of the regular Nvim Lua environment. That also meant os.exit() coverage had to carry an ASAN workaround because Lua's raw process exit skipped Nvim teardown and let LeakSanitizer interfere with the observed exit code. Solution: Run the harness and related fixtures with nvim -l. Patch os.exit() in the main Lua state to exit through getout(), so scripts observe normal Nvim shutdown while standalone -ll remains available for generator-style scripts. As a consequence, the startup test can assert os.exit() without disabling leak detection. AI-assisted: Codex
* ci: drop cirrus #39321Justin M. Keyes2026-04-22
| | | | | | | | Problem: cirrus will shutdown soon, and we are running out of minutes anyway, which causes ci failures. Solution: Drop cirrus config.
* test: replace busted with local harnessLewis Russell2026-04-15
| | | | | | | | | | | | | | | | | | | Replace the busted-based Lua test runner with a repo-local harness. The new harness runs spec files directly under `nvim -ll`, ships its own reporter and lightweight `luassert` shim, and keeps the helper/preload flow used by the functional and unit test suites. Keep the file boundary model shallow and busted-like by restoring `_G`, `package.loaded`, `package.preload`, `arg`, and the process environment between files, without carrying extra reset APIs or custom assertion machinery. Update the build and test entrypoints to use the new runner, add black-box coverage for the harness itself, and drop the bundled busted/luacheck dependency path. AI-assisted: Codex
* test: start test runners in Xtest_xdg dir (#37964)zeertzjq2026-02-20
| | | | | | | | | | | | | | This is a better way to prevent parallel tests from interfering with each other, as there are many ways files can be created and deleted in tests, so enforcing different file names is hard. Using $TMPDIR can also work in most cases, but 'backipskip' etc. have special defaults for $TMPDIR. Symlink runtime/, src/, test/ and README.md to Xtest_xdg dir to make tests more convenient (and symlinking test/ is required for busted). Also, use README.md instead of test/README.md in the Ex mode inccommand test, as test/README.md no longer contains 'N' char.
* test: remove XDG dir and rplugin manifest on cleanup (#37954)zeertzjq2026-02-18
|
* feat(logging): rename ".nvimlog" => "nvim.log" #37935Justin M. Keyes2026-02-18
| | | | | | - Rename ".nvimlog" to "nvim.log" - doesn't need to be "hidden"/dotfile - ".log" extension helps with filetype detection - Also rename "nvim/log" => "nvim/nvim.log"
* test: support running functionaltests in parallel by directory (#37918)zeertzjq2026-02-18
| | | | | | | | | Define a CMake target for every subdirectory of test/functional that contains functional tests, and a functionaltest_parallel target that depends on all those targets, allowing multiple test runners to run in parallel. On CI, use at most 2 parallel test runners, as using more may increase system load and make tests unstable.
* test: fail CI if .nvimlog is not empty (#37470)zeertzjq2026-01-20
| | | | | | | | This will prevent adding passing tests which logs errors in .nvimlog. Close #28323 Related: https://github.com/neovim/neovim/pull/27660#issuecomment-2035738597 Co-authored-by: dundargoc <gocdundar@gmail.com>
* build: ship "_core/*" as bytecode (built-into Nvim binary)Justin M. Keyes2025-12-30
| | | | | | | | | | | | | | | | | | | | | | | | | | Problem: We want to encourage implementing core features in Lua instead of C, but it's clumsy because: - Core Lua code (built into `nvim` so it is available even if VIMRUNTIME is missing/invalid) requires manually updating CMakeLists.txt, or stuffing it into `_editor.lua`. - Core Lua modules are not organized similar to C modules, `_editor.lua` is getting too big. Solution: - Introduce `_core/` where core Lua code can live. All Lua modules added there will automatically be included as bytecode in the `nvim` binary. - Move these core modules into `_core/*`: ``` _defaults.lua _editor.lua _options.lua _system.lua shared.lua ``` TODO: - Move `_extui/ => _core/ui2/`
* test: skip noisy "terminal mode unavailable" logs #36794Justin M. Keyes2025-12-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Every CI log has a lot of noise at the end, which makes it harder to find relevant test failures: Running tests from test/functional/terminal/tui_spec.lua ... T5831 TUI bg color queries the terminal for background color: T5832 TUI bg color triggers OptionSet from automatic background processing: T5833 TUI bg color sends theme update notifications when background changes #31652: ... Running tests from test/functional/ui/output_spec.lua ... WRN 2025-12-02T03:36:47.304 ui/c/T5831.28003.0 tui_handle_term_mode:223: TUI: terminal mode 2026 unavailable, state 0 WRN 2025-12-02T03:36:47.359 ui/c/T5832.28006.0 tui_handle_term_mode:223: TUI: terminal mode 2048 unavailable, state 0 WRN 2025-12-02T03:36:47.414 ui/c/T5833.28009.0 tui_handle_term_mode:223: TUI: terminal mode 2048 unavailable, state 0 Solution: - Skip logging in test-mode. - This can be reverted later, when these logs are changed to "INFO" level, per this TODO comment: ``` // TODO(bfredl): This is really ILOG but we want it in all builds. // add to show_verbose_terminfo() without being too racy ???? WLOG("TUI: terminal mode %d unavailable, state %d", mode, state); ```
* build: unset $TMUX when running tests (#34178)zeertzjq2025-05-27
| | | | | | This prevents Nvim TUI running in tests from thinking it's inside tmux. Another solution is make :terminal unset $TMUX instead, but I'm not sure if that'll break some other use cases.
* fix(tests): don't surpress stderr outputbfredl2025-05-22
| | | | | | | This is a diabolical anti-pattern and is hiding errors which exist currently in unittests. Also we want to see _where_ in the process stderr was emitted, stashing it away at the end erases important context.
* feat(build): build.zig MVP: build and run functionaltests on linuxbfredl2025-05-02
| | | | | | | | | | | | | | | | | | | | | NEW BUILD SYSTEM! This is a MVP implementation which supports building the "nvim" binary, including cross-compilation for some targets. As an example, you can build a aarch64-macos binary from an x86-64-linux-gnu host, or vice versa Add CI target for build.zig currently for functionaltests on linux x86_64 only Follow up items: - praxis for version and dependency bumping - windows 💀 - full integration of libintl and gettext (or a desicion not to) - update help and API metadata files - installation into a $PREFIX - more tests and linters
* build: move all generator scripts to `src/gen/`Lewis Russell2025-02-26
| | | | | | | | | | | - Move all generator Lua scripts to the `src/gen/` - Add a `.luarc.json` to `src/gen/` - Add a `preload.lua` to `src/gen/` - Add `src` to `package.path` so it aligns with `.luarc.json' - Fix all `require` statements in `src/gen/` so they are consistent: - `require('scripts.foo')` -> `require('gen.foo')` - `require('src.nvim.options')` -> `require('nvim.options')` - `require('api.dispatch_deprecated')` -> `require('nvim.api.dispatch_deprecated')`
* test: override XDG_STATE_HOME as well (#28604)zeertzjq2024-05-02
| | | This prevents tested Nvim from affecting ~/.local/state/
* test: improve test conventionsdundargoc2024-04-08
| | | | Work on https://github.com/neovim/neovim/issues/27004.
* fix(test): call separate_arguments() correctly to honor $BUSTED_ARGS (#26905)James McCoy2024-01-05
| | | | | separate_arguments() doesn't operate in place on a variable. It expects a string argument to parse and stores the results in the specified variable.
* build: cmake fixesdundargoc2023-12-16
| | | | | | | | | | | | | | - add EXTERNALPROJECT_OPTIONS variable to main build - use `REQUIRED` keyword for IWYU. - remove check_c_compiler_flag checks when `ENABLE_COMPILER_SUGGESTIONS` is `ON`. If we explicitly enable it then we probably want it to give an error if it doesn't exist, rather than silently skip it. - Move dependency interface libraries to their find module and use them as a pseudo-imported target. - Remove BUSTED_OUTPUT_TYPE. It's not used and we can reintroduce it again if something similar is needed. - Use LINK_OPTIONS intead of LINK_FLAGS when generating the `--version` output.
* fix(tests): set SHELL=sh #24941Sergey Slipchenko2023-09-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Some tests fail with $SHELL=fish #6172 Related: https://github.com/neovim/neovim/pull/6176 Solution: Replace "echo -n" with "printf", because "echo" in sh may be provided as a shell builtin, which does not accept an "-n" flag to avoid a trailing newline (e.g. on macos). "printf" is more portable (defined by POSIX) and it does not output a trailing newline by itself. Fixes #6172 TODO: Other test failures may be related to "session leader" issue: https://github.com/neovim/neovim/issues/2354 Checked by running `:terminal ./build/bin/tty-test` from Nvim with `shell=/bin/fish` (inherited from `$SHELL`) and it indeed complains about "process does not own the terminal". With `shell=sh` it doesn't complain. And unsetting `$SHELL` seems to make `nvim` to fall back to `shell=sh`. FAILED test/functional/terminal/tui_spec.lua @ 1017: TUI paste: terminal mode test/functional/terminal/tui_spec.lua:1024: Row 1 did not match. Expected: |*tty ready | |*{1: } | |* | | | |{5:^^^^^^^ }| |{3:-- TERMINAL --} | |{3:-- TERMINAL --} | Actual: |*process does not own the terminal | |* | |*[Process exited 2]{1: } | | | |{5:^^^^^^^ }| |{3:-- TERMINAL --} | |{3:-- TERMINAL --} | To print the expect() call that would assert the current screen state, use screen:snapshot_util(). In case of non-deterministic failures, use screen:redraw_debug() to show all intermediate screen states. stack traceback: test/functional/ui/screen.lua:622: in function '_wait' test/functional/ui/screen.lua:352: in function 'expect' test/functional/terminal/tui_spec.lua:1024: in function <test/functional/terminal/tui_spec.lua:1017> FAILED test/functional/terminal/tui_spec.lua @ 1551: TUI forwards :term palette colors with termguicolors test/functional/terminal/tui_spec.lua:1567: Row 1 did not match. Expected: |*{1:t}ty ready | | | |* | | | |{2:^^^^^^^ }| | | |{3:-- TERMINAL --} | Actual: |*{1:p}rocess does not own the terminal | | | |*[Process exited 2] | | | |{2:^^^^^^^ }| | | |{3:-- TERMINAL --} | To print the expect() call that would assert the current screen state, use screen:snapshot_util(). In case of non-deterministic failures, use screen:redraw_debug() to show all intermediate screen states. stack traceback: test/functional/ui/screen.lua:622: in function '_wait' test/functional/ui/screen.lua:352: in function 'expect' test/functional/terminal/tui_spec.lua:1567: in function <test/functional/terminal/tui_spec.lua:1551>
* refactor(build): derocksify luacheckbfredl2023-09-07
|
* feat(version): unverbose ":version", ":verbose version" #24195Justin M. Keyes2023-07-01
| | | | | | | | | Problem: `nvim -v` and `:version` prints system vimrc, fallback files, and compilation info by default, which most people don't care about and just clutters up the output. Solution: Omit extra info unless 'verbose' is set.
* build: move luarocks and rocks installation to main builddundargoc2023-05-21
| | | | | | This will ensure luacheck and busted are only installed when they're actually needed. This cuts total build time by over 50%. Closes https://github.com/neovim/neovim/issues/22797.
* build: cmake cleanupdundargoc2023-05-13
| | | | | | | | | | - Simplify error checking when using execute_process. - Set BUILD_SHARED_LIBS to OFF when building dependencies. This is normally not needed, but msgpack interprets an unset BUILD_SHARED_LIBS to build a shared library, which is the opposite of the cmake behavior. - Move function check_lua_module to Util.cmake. - Remove unnecessary code. - Make variable naming more consistent
* build: don't check environment variable to detect CI (#22234)dundargoc2023-02-12
| | | | Instead use the cmake option, which should act as the definitive source to determine whether we use CI or not.
* refactor(tests): move lua-client into core and use it for functionaltestsbfredl2023-02-10
| | | | | | | | Eliminates lua-client and non-static libluv as test time dependencies Note: the API for a public lua-client is not yet finished. The interface needs to be adjusted to work in the embedded loop of a nvim instance (to use it to talk between instances)
* refactor(tests): run unittests using main nvim binary in interpreter modebfredl2023-01-31
| | | | This allows us to get rid of the separate "nvim-test" target
* ci: add cirrus to isCI function to skip tests (#20526)dundargoc2022-10-17
| | | | The environment variable CIRRUS_CI is manually passed to RunTests.cmake as it doesn't get passed when using cmake script mode.
* revert: "build: remove unnecessary policy related code" #20289dundargoc2022-09-23
| | | | | | | | | This partially reverts commit 42aeb5c5b18af1362362a2e6bdf10a2a4ec70f0f. Setting cmake policies is normally not required as cmake_minimum_required automatically sets these. One exception is cmake script mode (-P) since it automatically resets all policy changes. Closes: https://github.com/neovim/neovim/issues/20286
* build: remove unnecessary policy related codedundargoc2022-09-08
| | | | | Having cmake version 3.10 as the required minimum version ensures these are set to new by default.
* fix(tests): remove misleading $TEST_PATH segment #19050Justin M. Keyes2022-06-23
| | | | | | | | | | | | | | | Problem: RunTests.cmake adds $TEST_PATH to $TMPDIR with the implication that it gives more isolation. But this is misleading because $TEST_PATH is only defined once. Full test runs use the same $TMPDIR for all tests. This was likely added with the intention of invoking RunTests.cmake once-per-testfile from a wrapper than does the isolation/orchestration. But even so, Nvim's vim_maketempdir() / vim_mktempdir() _already_ creates a unique tempdir per session. Solution: Don't append $TEST_PATH to $TMPDIR. Avoids confusion and makes the path shorter.
* refactor(log): simplify log_path_init #18898Justin M. Keyes2022-06-08
| | | | | | | | | Problem: Since 22b52dd462e5 #11501, log_path_init is called in log_init, so it is now called at a deterministic time. So the "just in time" complexity of log_path_init is no longer needed. Solution: Remove logic intended to try to "heal" partial initialization.
* ci: show failed message on Windowszeertzjq2022-03-03
|
* ci: skip tests that fail on windowsDundar Göc2022-02-20
|
* test: allow excluding functional/unit tests using TEST_FILTER_OUTJames McCoy2021-12-18
| | | | | | | Although this can already be done using `BUSTED_ARGS`, it complements our existing shortcut of `TEST_FILTER.` [skip ci]
* build(tests): isolate the user environment XDG_DATA_DIRS #16003Javier Lopez2021-10-12
| | | | | | | | | | Problem: Some tests were not passing on my machine, specifically in `test/functional/api/vim_spec.lua` the two tests under `describe('nvim_get_runtime_file...` Solution: Unset `XDG_DATA_DIRS` in the test runner. Note: Window CI failed if we set it to the same value as `XDG_DATA_HOME`.
* Merge branch 'master' into histfileJustin M. Keyes2021-09-10
|\
| * test: use $TEST_TIMEOUT to specify timeoutJames McCoy2021-09-09
| |
* | do not write into user's ~/.bash_historyJun T2021-09-10
|/
* test: timeout at 20 minutes #15597Justin M. Keyes2021-09-08
| | | | | | | | | | | | | | | Set a maximum test run-time of 20 minutes to: - fail faster - avoid wasting CI resources - set a bound on the test time (if tests take longer than 20 minutes we need to invest in parallelizing them...) Timeout looks like: -- Tests exited non-zero: Process terminated due to timeout -- No output to stderr. CMake Error at /…/neovim/cmake/RunTests.cmake:86 (message): functional tests failed with error: Process terminated due to timeout
* ci: RunTests: ensure that the logfile gets displayed (#10597)Daniel Hahler2019-07-24
| | | | | | This is meant to help with test failures where e.g. `busted` itself might crash or get killed. Ref: https://github.com/neovim/neovim/issues/10182#issuecomment-514450069
* test: Force $TEST_FILE to relative path [ci skip] #10535erw72019-07-19
| | | | | - Always force TEST_PATH to a relative path. - Eliminate rel_test_path. (The code simplification is worth the extra subdirectory below Xtest_tmpdir/.)
* tests: move "busted" dir to "test" (#10518)Daniel Hahler2019-07-16
| | | | | | | | | | | * tests: move "busted" dir to "test" It is used for outputHandlers only, and clearly belongs to the tests. Use the full module name with the `-o` option to `busted` then for clarity. * luacheck * test/busted/outputHandlers/TAP.lua: use/extend upstream
* cmake/RunTests.cmake: fix TEST_TAG/TEST_FILTER [ci skip] #10371Daniel Hahler2019-06-29
| | | | | Followup to f1f9a2f97. The mentioned example there did not work after all, likely due to changing quoting during PR review.
* CMakeLists: remove/cleanup passing of CMAKE_SYSTEM_NAME (#10351)Daniel Hahler2019-06-27
| | | | | | | | Just set it from `${CMAKE_HOST_SYSTEM_NAME}` directly, instead of passing it from the main CMake file (CMAKE_SYSTEM_NAME defaults to it, but is empty in script mode). Initially added in 9ce81f7b2, but then even used with unrelated commands (that do not use RunTests.cmake, e.g. 221f6fffa).
* tests: busted: do not use "--lua" (#10303)Daniel Hahler2019-06-24
| | | | | | | | | | | | | | The feature is very rudimentary [1], and causes problems when not using bundled deps, where it might then fail to find `busted.runner` due to this. E.g. with "-DUSE_BUNDLED=OFF -DUSE_BUNDLED_LUAROCKS=ON": `.deps/usr/bin/busted` contains `exec '/usr/bin/luajit' -e …`, i.e. it run luajit with adjusted lpath etc. But using `--lua /usr/bin/luajit` then causes this `busted` wrapper to be replaced with just `/usr/bin/luajit`, i.e. it is missing the adjusted lpath then. 1: https://github.com/Olivine-Labs/busted/commit/9eb6c9cf93bc558a2398c822b896990e48bd69fd
* tests: improve RunTests.cmake (#10239)Daniel Hahler2019-06-22
| | | | | | | | | | | | | | | | - allow for passing in BUSTED_ARGS via env - quote values of TEST_TAG/TEST_FILTER Previously TEST_FILTER="'foo bar'" was required. This allows for: make functionaltest TEST_FILE=test/functional/terminal/tui_spec.lua \ BUSTED_ARGS="--no-keep-going --shuffle" \ TEST_FILTER="TUI background color handles" * RunTests.cmake: BUILD_DIR for Xtest files, isolated TMPDIR Assume relative path for given TEST_PATH.
* runtime/lua/vim/compat.luaJustin M. Keyes2018-12-20
| | | | | | | | | | | | | | | | | | | | | ref #9280 Introduce the `vim.compat` module, to help environments with system Lua 5.2+ run the build/tests. Include the module implicitly in all tests. ref #8677 legacy `vim` module: beep buffer command dict eval firstline lastline line list open type window
* build: adjust some messages (#9087)Justin M. Keyes2018-10-06
|
generated by cgit (git 2.34.1) at 2026-10-19 17:53:34 +0000