aboutsummaryrefslogtreecommitdiff
path: root/test/unit/mbyte_spec.lua
Commit message (Collapse)AuthorAge
* fix(setreg): use better estimate of register widthbfredl9 days
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | vim-patch:9.2.1175: setreg() block width wrong for unprintable chars When yanking, the width of a <c-v>{motion}y register is determined by the virtual colums of the text. However in setreg() in 'b' mode use an idosyncratic width measurement where C0 control chars (ASCII 0x00 - 0x19) and invalid UTF-8 bytes have width 1 instead of their display width (2 or 4 cells). Consider the following test case: let strings = ['"reference', '"Übel', "\"aa\x07bb", "\"qq\x82", "\"x\nbabbel"] for string in strings call append('$', string) exec "normal G0\<c-v>$\"ay" let one = getregtype('a') if string != strings[-1] call setreg("b", string, 'b') let two = getregtype('b') else let two = "na" endif call setreg("c", [string], 'b') let three = getregtype('c') call append('$', '"yank: '.one.", string: ".two.", list: ".three) endfor AFter this fix, all the three ways of measuring the width gives equal values (or two, in the case of not being able to use setreg() with a plain string in the first case)
* refactor(mbyte): use StrCharInfo/ClusterInfo also for str[len] casesbfredl2026-10-06
| | | | | | | | part of a series, next big one is gonna be bring utf_ptr2schar and friends into the New world. after that it might be worth brushing of some benchmarks and re-microoptimize the hottest loops. also some const-y constess but it is fine.
* fix(mbyte): spacing marks need their own display cellwoot2026-09-15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: strdisplaywidth('ทำ') is 1, but that cluster needs two cells. A SpacingMark has positive advance width (Unicode core spec D55), so it cannot share its base's cell: no Thai font fits a consonant plus SARA AM (U+0E33) in one cell, the narrowest needing 119% of one and most 180-210%, measured against a 0.600 em cell. UAX#29 GB9a keeps the mark in its base's cluster, which is correct for segmentation and says nothing about width. setcellwidths() cannot work around it, because the collapse is per cluster, not per codepoint. Vim reports 2 here. Devanagari and Bengali matras and Khmer and Tamil vowel signs are the same case. Solution: Add a cell for each remaining codepoint in the cluster whose boundclass is UTF8PROC_BOUNDCLASS_SPACINGMARK, clamped to the maximum cells per cluster. Segmentation is untouched, so the cursor and selection still treat the cluster as one unit. U+FF9E and U+FF9F, the halfwidth katakana sound marks, are Grapheme_Extend rather than SpacingMark, so they are matched separately. This is not a general solution for grapheme extenders that carry width -- that needs a queryable table -- but it does fix the reported regressions for Thai and Japanese (refs #34733). AI-assisted fixes #34733
* test: fix wrong order of args to eq(), neq() #41499zeertzjq2026-08-26
|
* test(harness): migrate away from magic globalsJustin M. Keyes2026-07-21
| | | | | | | | | | | | | | | | | | | | | Problem: The magic globals `it`, `describe`, etc., are more trouble than they are worth. - Hooking into `after_each` requires `getfenv()` hacks. - They confuse luals/emmylua, because the top-level `.luarc.json` isn't merged with `test/.luarc.json` (apparently a luals limitation?) - They totally defeat discoverability because the user just has to "know" about the various magic symbols. So they harm DX, which means they serve no purpose at all. Solution: - Expose the test API from `testutil`, so tests can call `t.it()`, `t.describe()`, etc., in the conventional way. - Drop `getfenv()` hacks. - Drop the `setfenv()` injection in `load_chunk`. - Drop `test/_meta.lua`.
* docs: miscdundargoc2025-01-11
| | | | | | | | | | | | | | | Co-authored-by: Axel <axelhjq@gmail.com> Co-authored-by: Colin Kennedy <colinvfx@gmail.com> Co-authored-by: Daiki Noda <sys9kdr@users.noreply.github.com> Co-authored-by: Evgeni Chasnovski <evgeni.chasnovski@gmail.com> Co-authored-by: Jean-Jacq du Plessis <1030058+jj-du-plessis@users.noreply.github.com> Co-authored-by: Juan Giordana <juangiordana@gmail.com> Co-authored-by: Lincoln Wallace <locnnil0@gmail.com> Co-authored-by: Matti Hellström <hellstrom@scm.com> Co-authored-by: Steven Locorotondo <steven.locorotondo@justeattakeaway.com> Co-authored-by: Yochem van Rosmalen <git@yochem.nl> Co-authored-by: glepnir <glephunter@gmail.com> Co-authored-by: ifish <fishioon@live.com>
* docs: misc (#30177)dundargoc2024-09-29
| | | | | Co-authored-by: Christian Clason <c.clason@uni-graz.at> Co-authored-by: Riley Bruins <ribru17@hotmail.com> Co-authored-by: zeertzjq <zeertzjq@outlook.com>
* refactor(multibyte): neo-casefolding without allocationbfredl2024-09-29
| | | | fixes #30400
* fix(mbyte): check for utf8proc_map() failure (#30531)zeertzjq2024-09-26
|
* fix(multibyte): handle backspace of wide clusters in replace modebfredl2024-09-06
| | | | | Make utf_head_off more robust against invalid sequences and embedded NUL chars
* feat(mbyte): support extended grapheme clusters including more emojibfredl2024-08-30
| | | | | | | | | Use the grapheme break algorithm from utf8proc to support grapheme clusters from recent unicode versions. Handle variant selector VS16 turning some codepoints into double-width emoji. This means we need to use ptr2cells rather than char2cells when possible.
* refactor(test): inject after_each differentlyLewis Russell2024-04-10
|
* test: improve test conventionsdundargoc2024-04-08
| | | | Work on https://github.com/neovim/neovim/issues/27004.
* fix(mbyte): fix bugs in utf_cp_*_off() functionsVanaIgr2024-02-26
| | | | | | Problems: - Illegal bytes after valid UTF-8 char cause utf_cp_*_off() to fail. - When stream isn't NUL-terminated, utf_cp_*_off() may go over the end. Solution: Don't go over end of the char of end of the string.
* build: enable lintlua for test/unit/ dir #26396Justin M. Keyes2023-12-04
| | | | | | | | | | | | | Problem: Not all Lua code is checked by stylua. Automating code-style is an important mechanism for reducing time spent on accidental (non-essential) complexity. Solution: - Enable lintlua for `test/unit/` directory. - TODO: only `test/functional/` remains unchecked. previous: 45fe4d11add933df76a2ea4bf52ce8904f4a778b previous: 517f0cc634b985057da5b95cf4ad659ee456a77e
* refactor(grid): make screen rendering more multibyte than ever beforebfredl2023-11-17
| | | | | | | | | | | | | | | | | | Problem: buffer text with composing chars are converted from UTF-8 to an array of up to seven UTF-32 values and then converted back to UTF-8 strings. Solution: Convert buffer text directly to UTF-8 based schar_T values. The limit of the text size is now in schar_T bytes, which is currently 31+1 but easily could be raised as it no longer multiplies the size of the entire screen grid when not used, the full size is only required for temporary scratch buffers. Also does some general cleanup to win_line text handling, which was unnecessarily complicated due to multibyte rendering being an "opt-in" feature long ago. Nowadays, a char is just a char, regardless if it consists of one ASCII byte or multiple bytes.
* build/doc/CI: remove/update quickbuild references #11258Justin M. Keyes2019-10-19
|
* test/mbyte_spec: skip broken test on QuickBuildJustin M. Keyes2019-08-06
| | | | Forgot `return` in eec529cf9e13.
* test/mbyte_spec: skip broken test on QuickBuildJustin M. Keyes2019-08-05
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 14:13:04,119 INFO - # ./test/unit/helpers.lua @ 760: mbyte utf_char2bytes for chars 0xa000 - 0xafff 14:13:06,307 WARN - E908: using an invalid value as a String /usr/home/quickbuild/buildagent/workspace/root/neovim/pull-requests-automated/.deps/usr/bin/luajit: ./test/unit/helpers.lua:459: write() error: 32: Broken pipe 14:13:06,308 WARN - stack traceback: 14:13:06,308 WARN - [C]: in function 'throw' 14:13:06,308 WARN - ...quests-automated/.deps/usr/share/lua/5.1/busted/core.lua:149: in function 'error' 14:13:06,308 WARN - ...ts-automated/.deps/usr/share/lua/5.1/luassert/assert.lua:171: in function 'assert' 14:13:06,308 WARN - ./test/unit/helpers.lua:459: in function 'write' 14:13:06,308 WARN - ./test/unit/helpers.lua:626: in function 'hook' 14:13:06,308 WARN - ./test/unit/helpers.lua:574: in function <./test/unit/helpers.lua:557> 14:13:06,308 WARN - [C]: in function 'type' 14:13:06,308 WARN - ...d/.deps/usr/share/lua/5.1/busted/outputHandlers/base.lua:57: in function 'copyElement' 14:13:06,308 WARN - ...d/.deps/usr/share/lua/5.1/busted/outputHandlers/base.lua:66: in function 'format' 14:13:06,308 WARN - ...d/.deps/usr/share/lua/5.1/busted/outputHandlers/base.lua:172: in function 'fn' 14:13:06,308 WARN - ...-requests-automated/.deps/usr/share/lua/5.1/mediator.lua:103: in function 'publish' 14:13:06,308 WARN - ...quests-automated/.deps/usr/share/lua/5.1/busted/core.lua:201: in function 'safe' 14:13:06,308 WARN - ...quests-automated/.deps/usr/share/lua/5.1/busted/core.lua:312: in function 'execute' 14:13:06,308 WARN - ...sts-automated/.deps/usr/share/lua/5.1/busted/execute.lua:58: in function 'execute' 14:13:06,308 WARN - ...ests-automated/.deps/usr/share/lua/5.1/busted/runner.lua:197: in function <...ests-automated/.deps/usr/share/lua/5.1/busted/runner.lua:11> 14:13:06,308 WARN - ./.deps/usr/lib/luarocks/rocks/busted/2.0.0-1/bin/busted:3: in main chunk 14:13:06,308 WARN - [C]: at 0x004041a0 14:13:06,323 WARN - Terminated 14:13:06,325 INFO - Executing post-execute action... 14:13:06,526 INFO - Checking step execute condition... 14:13:06,526 INFO - Step execute condition satisfied, executing... 14:13:06,706 INFO - Executing pre-execute action... 14:13:06,706 INFO - Running step...
* unit/mbyte_spec: Run utf_char2bytes test in batches of 0xFFF charactersJames McCoy2018-11-15
| | | | | Running the full 0xFFFF set of tests in one process fails on slower architectures/when there's CPU contention.
* test: port kword_test to Lua for utf_char2bytes()Jan Edmund Lazo2018-07-01
| | | | | | Use LuaJIT FFI to create char pointer. Validate output with utf_ptr2char(), vim_iswordc() and vim_iswordp(). Use const for LuaJIT string-to-char conversion.
* unittests: Check core dumps in after_each, like in functestsZyX2017-03-11
|
* unittests: Run all unit tests in their own processesZyX2017-03-11
| | | | | | | | | | | | | | | Used sed -r -i -e '/ helpers =/ s/$/\nlocal itp = helpers.gen_itp(it)/; s/^(\s*)it\(/\1itp(/' test/unit/**/*_spec.lua to alter all tests. Locally they all run fine now. Reasoning: 1. General: state from one test should not affect other tests. 2. Local: travis build is failing with something which may be an output of garbage collector. This should prevent state of the garbage collector from interferring as well.
* Satisfy testlint.KillTheMule2016-04-28
| | | | For that, make luatest ignore the preload.lua files.
* mbyte_spec.lua: Fix indentationoni-link2016-04-17
|
* mbyte_spec.lua: Fix wordingoni-link2016-04-17
|
* mbyte.c: Unittest for utfc_ptr2char_len()oni-link2016-04-15
generated by cgit (git 2.34.1) at 2026-10-20 15:58:15 +0000