aboutsummaryrefslogtreecommitdiff
path: root/cmake/RunTests.cmake
Commit message (Collapse)AuthorAge
* fix(tests): set SHELL=sh #24941Sergey Slipchenko2023-09-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Some tests fail with $SHELL=fish #6172 Related: https://github.com/neovim/neovim/pull/6176 Solution: Replace "echo -n" with "printf", because "echo" in sh may be provided as a shell builtin, which does not accept an "-n" flag to avoid a trailing newline (e.g. on macos). "printf" is more portable (defined by POSIX) and it does not output a trailing newline by itself. Fixes #6172 TODO: Other test failures may be related to "session leader" issue: https://github.com/neovim/neovim/issues/2354 Checked by running `:terminal ./build/bin/tty-test` from Nvim with `shell=/bin/fish` (inherited from `$SHELL`) and it indeed complains about "process does not own the terminal". With `shell=sh` it doesn't complain. And unsetting `$SHELL` seems to make `nvim` to fall back to `shell=sh`. FAILED test/functional/terminal/tui_spec.lua @ 1017: TUI paste: terminal mode test/functional/terminal/tui_spec.lua:1024: Row 1 did not match. Expected: |*tty ready | |*{1: } | |* | | | |{5:^^^^^^^ }| |{3:-- TERMINAL --} | |{3:-- TERMINAL --} | Actual: |*process does not own the terminal | |* | |*[Process exited 2]{1: } | | | |{5:^^^^^^^ }| |{3:-- TERMINAL --} | |{3:-- TERMINAL --} | To print the expect() call that would assert the current screen state, use screen:snapshot_util(). In case of non-deterministic failures, use screen:redraw_debug() to show all intermediate screen states. stack traceback: test/functional/ui/screen.lua:622: in function '_wait' test/functional/ui/screen.lua:352: in function 'expect' test/functional/terminal/tui_spec.lua:1024: in function <test/functional/terminal/tui_spec.lua:1017> FAILED test/functional/terminal/tui_spec.lua @ 1551: TUI forwards :term palette colors with termguicolors test/functional/terminal/tui_spec.lua:1567: Row 1 did not match. Expected: |*{1:t}ty ready | | | |* | | | |{2:^^^^^^^ }| | | |{3:-- TERMINAL --} | Actual: |*{1:p}rocess does not own the terminal | | | |*[Process exited 2] | | | |{2:^^^^^^^ }| | | |{3:-- TERMINAL --} | To print the expect() call that would assert the current screen state, use screen:snapshot_util(). In case of non-deterministic failures, use screen:redraw_debug() to show all intermediate screen states. stack traceback: test/functional/ui/screen.lua:622: in function '_wait' test/functional/ui/screen.lua:352: in function 'expect' test/functional/terminal/tui_spec.lua:1567: in function <test/functional/terminal/tui_spec.lua:1551>
* refactor(build): derocksify luacheckbfredl2023-09-07
|
* feat(version): unverbose ":version", ":verbose version" #24195Justin M. Keyes2023-07-01
| | | | | | | | | Problem: `nvim -v` and `:version` prints system vimrc, fallback files, and compilation info by default, which most people don't care about and just clutters up the output. Solution: Omit extra info unless 'verbose' is set.
* build: move luarocks and rocks installation to main builddundargoc2023-05-21
| | | | | | This will ensure luacheck and busted are only installed when they're actually needed. This cuts total build time by over 50%. Closes https://github.com/neovim/neovim/issues/22797.
* build: cmake cleanupdundargoc2023-05-13
| | | | | | | | | | - Simplify error checking when using execute_process. - Set BUILD_SHARED_LIBS to OFF when building dependencies. This is normally not needed, but msgpack interprets an unset BUILD_SHARED_LIBS to build a shared library, which is the opposite of the cmake behavior. - Move function check_lua_module to Util.cmake. - Remove unnecessary code. - Make variable naming more consistent
* build: don't check environment variable to detect CI (#22234)dundargoc2023-02-12
| | | | Instead use the cmake option, which should act as the definitive source to determine whether we use CI or not.
* refactor(tests): move lua-client into core and use it for functionaltestsbfredl2023-02-10
| | | | | | | | Eliminates lua-client and non-static libluv as test time dependencies Note: the API for a public lua-client is not yet finished. The interface needs to be adjusted to work in the embedded loop of a nvim instance (to use it to talk between instances)
* refactor(tests): run unittests using main nvim binary in interpreter modebfredl2023-01-31
| | | | This allows us to get rid of the separate "nvim-test" target
* ci: add cirrus to isCI function to skip tests (#20526)dundargoc2022-10-17
| | | | The environment variable CIRRUS_CI is manually passed to RunTests.cmake as it doesn't get passed when using cmake script mode.
* revert: "build: remove unnecessary policy related code" #20289dundargoc2022-09-23
| | | | | | | | | This partially reverts commit 42aeb5c5b18af1362362a2e6bdf10a2a4ec70f0f. Setting cmake policies is normally not required as cmake_minimum_required automatically sets these. One exception is cmake script mode (-P) since it automatically resets all policy changes. Closes: https://github.com/neovim/neovim/issues/20286
* build: remove unnecessary policy related codedundargoc2022-09-08
| | | | | Having cmake version 3.10 as the required minimum version ensures these are set to new by default.
* fix(tests): remove misleading $TEST_PATH segment #19050Justin M. Keyes2022-06-23
| | | | | | | | | | | | | | | Problem: RunTests.cmake adds $TEST_PATH to $TMPDIR with the implication that it gives more isolation. But this is misleading because $TEST_PATH is only defined once. Full test runs use the same $TMPDIR for all tests. This was likely added with the intention of invoking RunTests.cmake once-per-testfile from a wrapper than does the isolation/orchestration. But even so, Nvim's vim_maketempdir() / vim_mktempdir() _already_ creates a unique tempdir per session. Solution: Don't append $TEST_PATH to $TMPDIR. Avoids confusion and makes the path shorter.
* refactor(log): simplify log_path_init #18898Justin M. Keyes2022-06-08
| | | | | | | | | Problem: Since 22b52dd462e5 #11501, log_path_init is called in log_init, so it is now called at a deterministic time. So the "just in time" complexity of log_path_init is no longer needed. Solution: Remove logic intended to try to "heal" partial initialization.
* ci: show failed message on Windowszeertzjq2022-03-03
|
* ci: skip tests that fail on windowsDundar Göc2022-02-20
|
* test: allow excluding functional/unit tests using TEST_FILTER_OUTJames McCoy2021-12-18
| | | | | | | Although this can already be done using `BUSTED_ARGS`, it complements our existing shortcut of `TEST_FILTER.` [skip ci]
* build(tests): isolate the user environment XDG_DATA_DIRS #16003Javier Lopez2021-10-12
| | | | | | | | | | Problem: Some tests were not passing on my machine, specifically in `test/functional/api/vim_spec.lua` the two tests under `describe('nvim_get_runtime_file...` Solution: Unset `XDG_DATA_DIRS` in the test runner. Note: Window CI failed if we set it to the same value as `XDG_DATA_HOME`.
* Merge branch 'master' into histfileJustin M. Keyes2021-09-10
|\
| * test: use $TEST_TIMEOUT to specify timeoutJames McCoy2021-09-09
| |
* | do not write into user's ~/.bash_historyJun T2021-09-10
|/
* test: timeout at 20 minutes #15597Justin M. Keyes2021-09-08
| | | | | | | | | | | | | | | Set a maximum test run-time of 20 minutes to: - fail faster - avoid wasting CI resources - set a bound on the test time (if tests take longer than 20 minutes we need to invest in parallelizing them...) Timeout looks like: -- Tests exited non-zero: Process terminated due to timeout -- No output to stderr. CMake Error at /…/neovim/cmake/RunTests.cmake:86 (message): functional tests failed with error: Process terminated due to timeout
* ci: RunTests: ensure that the logfile gets displayed (#10597)Daniel Hahler2019-07-24
| | | | | | This is meant to help with test failures where e.g. `busted` itself might crash or get killed. Ref: https://github.com/neovim/neovim/issues/10182#issuecomment-514450069
* test: Force $TEST_FILE to relative path [ci skip] #10535erw72019-07-19
| | | | | - Always force TEST_PATH to a relative path. - Eliminate rel_test_path. (The code simplification is worth the extra subdirectory below Xtest_tmpdir/.)
* tests: move "busted" dir to "test" (#10518)Daniel Hahler2019-07-16
| | | | | | | | | | | * tests: move "busted" dir to "test" It is used for outputHandlers only, and clearly belongs to the tests. Use the full module name with the `-o` option to `busted` then for clarity. * luacheck * test/busted/outputHandlers/TAP.lua: use/extend upstream
* cmake/RunTests.cmake: fix TEST_TAG/TEST_FILTER [ci skip] #10371Daniel Hahler2019-06-29
| | | | | Followup to f1f9a2f97. The mentioned example there did not work after all, likely due to changing quoting during PR review.
* CMakeLists: remove/cleanup passing of CMAKE_SYSTEM_NAME (#10351)Daniel Hahler2019-06-27
| | | | | | | | Just set it from `${CMAKE_HOST_SYSTEM_NAME}` directly, instead of passing it from the main CMake file (CMAKE_SYSTEM_NAME defaults to it, but is empty in script mode). Initially added in 9ce81f7b2, but then even used with unrelated commands (that do not use RunTests.cmake, e.g. 221f6fffa).
* tests: busted: do not use "--lua" (#10303)Daniel Hahler2019-06-24
| | | | | | | | | | | | | | The feature is very rudimentary [1], and causes problems when not using bundled deps, where it might then fail to find `busted.runner` due to this. E.g. with "-DUSE_BUNDLED=OFF -DUSE_BUNDLED_LUAROCKS=ON": `.deps/usr/bin/busted` contains `exec '/usr/bin/luajit' -e …`, i.e. it run luajit with adjusted lpath etc. But using `--lua /usr/bin/luajit` then causes this `busted` wrapper to be replaced with just `/usr/bin/luajit`, i.e. it is missing the adjusted lpath then. 1: https://github.com/Olivine-Labs/busted/commit/9eb6c9cf93bc558a2398c822b896990e48bd69fd
* tests: improve RunTests.cmake (#10239)Daniel Hahler2019-06-22
| | | | | | | | | | | | | | | | - allow for passing in BUSTED_ARGS via env - quote values of TEST_TAG/TEST_FILTER Previously TEST_FILTER="'foo bar'" was required. This allows for: make functionaltest TEST_FILE=test/functional/terminal/tui_spec.lua \ BUSTED_ARGS="--no-keep-going --shuffle" \ TEST_FILTER="TUI background color handles" * RunTests.cmake: BUILD_DIR for Xtest files, isolated TMPDIR Assume relative path for given TEST_PATH.
* runtime/lua/vim/compat.luaJustin M. Keyes2018-12-20
| | | | | | | | | | | | | | | | | | | | | ref #9280 Introduce the `vim.compat` module, to help environments with system Lua 5.2+ run the build/tests. Include the module implicitly in all tests. ref #8677 legacy `vim` module: beep buffer command dict eval firstline lastline line list open type window
* build: adjust some messages (#9087)Justin M. Keyes2018-10-06
|
* build/test: skip empty TEST_TAG, TEST_FILTERJustin M. Keyes2018-06-16
| | | | | - Empty "--tags=", "--filter=" prevents TEST_FILE from running. - Fix another bug: TEST_FILTER overwrote TEST_TAG.
* busted: explicit LUA_PATH to test scripts (#7864)Matthieu Coudron2018-01-20
| | | | in case LUA_PATH does not contain `./?.lua`, busted can not load lua test scripts. (for instance on nixos). Thus we make it explicit.
* test: force LC_ALL=en_US.UTF-8Justin M. Keyes2017-07-29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Tests that check localized error messages need a stable locale, else errors like this occur: [ FAILED ] 2 tests, listed below: [ FAILED ] ...npack/file/vim/neovim/test/functional/eval/null_spec.lua @ 29: NULL list is accepted as an empty list by writefile() ...npack/file/vim/neovim/test/functional/eval/null_spec.lua:30: Expected objects to be the same. Passed in: (string) ' E484: Cannot open file Xtest-functional-viml-null' Expected: (string) ' E484: Can't open file Xtest-functional-viml-null' stack traceback: ...npack/file/vim/neovim/test/functional/eval/null_spec.lua:30: in function <...npack/file/vim/neovim/test/functional/eval/null_spec.lua:29> [ FAILED ] ...k/file/vim/neovim/test/functional/ex_cmds/write_spec.lua @ 81: :write errors out correctly ...k/file/vim/neovim/test/functional/ex_cmds/write_spec.lua:97: Expected objects to be the same. Passed in: (string) 'Vim(write):E510: Cannot make backup file (add ! to override)' Expected: (string) 'Vim(write):E510: Can't make backup file (add ! to override)' stack traceback: ...k/file/vim/neovim/test/functional/ex_cmds/write_spec.lua:97: in function <...k/file/vim/neovim/test/functional/ex_cmds/write_spec.lua:81> 10 SKIPPED TESTS 2 FAILED TESTS -- Output to stderr: 2017/07/29 00:41:32 ERROR 31133/open_log_file:170: Logging to stderr, failed to open $NVIM_LOG_FILE: Xtest-startup-xdg-logpath/nvim/log 2017/07/29 00:41:32 WARN 31133/call_set_error:815: RPC: ch 1 was closed by the client CMake Error at /home/shlomif/Download/unpack/file/vim/neovim/cmake/RunTests.cmake:50 (message): Running functional tests failed with error: 1. FAILED: CMakeFiles/functionaltest cd /home/shlomif/Download/unpack/file/vim/neovim/build && /usr/bin/cmake -DBUSTED_PRG=/home/shlomif/Download/unpack/file/vim/neovim/.deps/usr/bin/busted -DLUA_PRG=/home/shlomif/Download/unpack/file/vim/neovim/.deps/usr/bin/luajit -DNVIM_PRG=/home/shlomif/Download/unpack/file/vim/neovim/build/bin/nvim -DWORKING_DIR=/home/shlomif/Download/unpack/file/vim/neovim -DBUSTED_OUTPUT_TYPE=nvim -DTEST_DIR=/home/shlomif/Download/unpack/file/vim/neovim/test -DBUILD_DIR=/home/shlomif/Download/unpack/file/vim/neovim/build -DTEST_TYPE=functional -DSYSTEM_NAME=Linux -P /home/shlomif/Download/unpack/file/vim/neovim/cmake/RunTests.cmake ninja: build stopped: subcommand failed. Makefile:102: recipe for target 'functionaltest' failed make: *** [functionaltest] Error 1
* test: Set $NVIM_LOG_FILE to test-local pathJustin M. Keyes2017-06-07
| | | | | | - Do not delete it: may need to inspect it after tests finished. - Avoids writing to stderr in cases where the test-local $XDG_DATA_HOME was not created yet.
* test: Use workspace-local temp directory.Justin M. Keyes2017-03-30
| | | | Closes #6291
* build: `oldtest` target: TEST_FILE, NVIM_PRG (#6098)Justin M. Keyes2017-02-12
| | | | | | | | | | | | | | | | | | | | | | - Add support for TEST_FILE to the `oldtest` target, for consistency with the busted/lua tests. Caveat: with the busted/lua tests TEST_FILE takes a full path, whereas for `oldtest` it must be "test_foo.res". - Add support for NVIM_PRG, again so that all test-related targets are consistent. - Use consistent name for NVIM_PRG. But still need to support NVIM_PROG for QuickBuild CI. Note: The `oldtest` target is driven by the top-level Makefile, because it requires a TTY. CMake 3.2 added a USES_TERMINAL flag to add_custom_target(). But we support CMake 2.8... add_custom_target(oldtest COMMAND make clean COMMAND make NVIM_PRG=$<TARGET_FILE:nvim> $ENV{MAKEOVERRIDES} DEPENDS nvim WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/src/nvim/testdir" USES_TERMINAL true )
* build: Do not force busted into front of $PATHJustin M. Keyes2017-02-04
| | | | | | | | | This was a workaround from long ago, but it doesn't seem to be needed anymore. And it breaks the $PATH on the Windows build (AppVeyor CI). After this change python3 (and 2) is correctly detected on AppVeyor CI. References #5946
* functionaltest: Create lua helper for os.tmpname()Rui Abreu Ferreira2016-08-31
| | | | | | | | | | | | | | | In Windows Lua's os.tmpname() returns relative paths starting with \s, prepend them with $TEMP to generate a valid path. In OS X os.tmpname() returns paths in '/tmp' but they should be in '/private/tmp'. We cannot use os_name() for platform detection because some tests use tempname() before nvim is spawned, instead use one of the following: 1. Set SYSTEM_NAME environment variable before calling the tests, it is set from CMAKE_SYSTEM_NAME(i.e. uname -s or 'Windows') 2. Call uname -s 3. Assume windows
* tests: Set env variables in RunTests.cmakeTommy Allen2016-08-17
| | | | | | - NVIM_RPLUGIN_MANIFEST - XDG_CONFIG_HOME - XDG_DATA_HOME
* Pass busted the path to the detected Lua interpreterJosh Triplett2016-07-13
| | | | | | | Otherwise, busted may run a different interpreter than the one we detected, without the capabilities we expect. (For instance, we might have detected the luajit interpreter, but busted might run the lua interpreter, without the ffi module.)
* test: Improve functional test debuggability and efficiencyThiago de Arruda2015-03-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | - Read TEST_TAG/TEST_FILTER env vars from cmake/RunTests.cmake. Setting these environment variables will pass --tags/--filter to busted, which can used to filter which tests are executed. - Remove calls to nvim msgpack-rpc API outside tests. This removes the requirement of having a static `clear` call in test/functional/helpers.lua - Use the new busted command-line option "--lazy" to ensure the setup/teardown hooks are only executed when a suite runs at least one test. Now its possible to run/debug a single test like this: ```sh TEST_FILTER='some test string' make test ``` Which will only run tests containing "some test string" in the title. Another option is: ```sh TEST_TAG=some-tag make test ``` After putting #some-tag into the test title. This also improves debugging experience because there will be no unnecessary gdbserver instances whe GDB=1 is passed.
* build: add support for running the tests in junit formatJohn Szakmeister2015-03-21
| | | | | | This requires a couple of extra modules that are not installed by default, and it requires capturing stdout of the tests--otherwise CMake output is intermixed with the XML output of busted.
* tests: prevent busted from reloading the ffi module and othersJohn Szakmeister2015-03-01
| | | | | | | | | | | It turns out that Busted started cleaning the environment in 2.0rc5 as a result of Olivine-Labs/busted#62. This, in turn, caused the ffi module to be reloaded for each spec file, and LuaJIT doesn't appreciate it. The net effect is an assertion error in LuaJIT. By using the --helper feature of Busted, we can pre-load some modules ahead of Busted and prevent it from reloading them--making LuaJIT happy again.
* CMake: Hide stderr output for successful test runs.Florian Walch2015-02-06
| | | | Fixes #1301.
* build: print the error result when the tests failJohn Szakmeister2014-11-25
| | | | | Any diagnostic information is useful when things fail. In my case, it printed out the fact that the tests were segfaulting.
* test: Set VIMRUNTIME before running testsThiago de Arruda2014-11-05
| | | | This is required for testing scripts under the 'runtime' directory
* CMake: Fix running individual (functional/unit) test.Florian Walch2014-11-05
|
* build: fix running of functional tests directly with CMakeJohn Szakmeister2014-11-05
| | | | | | | If you aren't just building everything into build/, then the functional tests fail because they can't find the nvim executable. Let's pass in the location of the nvim executable, and set NVIM_PRG environment variable accordingly.
* test: Remove run-functional-tests.pyThiago de Arruda2014-10-16
| | | | | | Now that the lua client is available, python/lupa are no longer necessary to run the functional tests. The helper functions previously defined in run-functional-tests.py were adapted to test/functional/helpers.lua.
* test: Move 'test/legacy' to 'test/functional'Thiago de Arruda2014-10-01
| | | | | | Busted can only discover tests from a single directory. In order to allow tests under 'legacy' to run as a functional test, it needed to be moved to 'test/functional'.
generated by cgit (git 2.34.1) at 2026-02-23 22:14:06 +0000