| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: Option validators and setters repeatedly pass error buffers and
their sizes separately. Some setters return messages in static storage.
Solution:
- Add CharBuf for borrowed writable storage, with its length including
space for a terminating NUL.
- Move option error buffers to the callers that report errors. Preserve
existing buffer sizes, truncation, and calls that do not need a
message.
- Consolidate option error formatting and the duplicated code that lists
valid option values.
- Return error messages directly from do_one_set_option().
- Remove unused arguments from get_option_newval() and
set_option_value_for().
- Handle legacy TTY assignments in the existing callers and remove the
redundant compatibility wrapper.
- Reuse CharBuf for Tree-sitter query errors.
- Pass CharBuf through the statusline and statuscolumn builders,
retaining the buffer sizes provided by their callers, including title
formatting.
AI-assisted
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
All wasm parsers shared a single global TSWasmStore via
ts_parser_set_wasm_store(). A TSWasmStore is owned by exactly one parser:
ts_parser_delete() frees it through ts_wasm_store_delete(). So the first
wasm parser collected freed the shared store, leaving the global pointer
dangling, and creating the next wasm parser dereferenced freed memory in
ts_wasm_store_reset() -> wasmtime_store_context(), crashing with SIGSEGV.
Give each parser its own store via ts_wasm_store_new() instead. This is
the 1:1 store-per-parser model intended by tree-sitter (see
tree-sitter/tree-sitter#3454): the global store remains only as the
language loader, ts_parser_delete() cleanly frees each parser's own
store, and the wasm engine is shared safely because ts_wasm_store_new()
clones its engine reference internally.
AI-assisted: Claude Code
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
parser_gc() calls ts_parser_delete() but leaves the userdata pointer
pointing to freed memory. If the GC finalizer runs at an unexpected time
(e.g. inside nvim_buf_get_lines #39411), a stale pointer could cause a crash.
Solution:
- NULL out `*ud` after ts_parser_delete() in parser_gc()
- Update parser_check() to handle NULL with a clear error message,
guarding all parser methods against UAF
Co-authored-by: Lewis Russell <lewis6991@gmail.com>
Signed-off-by: Szymon Wilczek <swilczek.lx@gmail.com>
|
| |
|
|
|
| |
Problem: Treesitter highlighting regressed on 32-bit builds because ranges that should cover the whole buffer were corrupted when passed into Lua.
Solution: Round-trip those range values through Lua and validate them so treesitter sees the same ranges on 32 and 64-bit builds.
|
| |
|
|
|
|
| |
This PR creates a C function `nts_parser_parse_buf()`
which is like `ts_parser_parse_string()` but instead can be passed
an nvim buffer number to parse.
|
| |
|
| |
This reverts commit 2a7cb32959b4c616bd2c76ae1933f8e068e391ad.
|
| |
|
|
|
|
|
|
|
|
| |
This commit changes `languagetree.lua` so that it creates a scratch
buffer under the hood when dealing with string parsers. This will make
it much easier to just use extmarks whenever we need to track injection
trees in `languagetree.lua`. This also allows us to remove the
`treesitter.c` code for parsing a string directly.
Note that the string parser's scratch buffer has `set noeol nofixeol` so
that the parsed source exactly matches the passed in string.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
**Problem(?):** Buffers that (for whatever reason) aren't meant to have
a final newline are still parsed with a final newline in `treesitter.c`.
**Solution:** Don't add the newline to the last buffer line if it
shouldn't be there. (This more closely matches the approach of
`read_buffer_into()`.)
This allows us to, say, use a scratch buffer with `noeol` and `nofixeol`
behind the scenes in `get_string_parser()`.
...which would allow us to track injection trees with extmarks in that
case.
...which would allow us to not drop previous trees after reparsing a
different range with `get_parser():parse()`.
...which would prevent flickering when editing a buffer that has 2+
windows to it in view at a time.
...which would allow us to keep our sanity!!!
(one step at a time...)
|
| | |
|
| |
|
|
|
|
|
|
|
| |
These are not needed after #35129 but making uncrustify still play nice
with them was a bit tricky.
Unfortunately `uncrustify --update-config-with-doc` breaks strings
with backslashes. This issue has been reported upstream,
and in the meanwhile auto-update on every single run has been disabled.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
The previous fix in #34314 relies on copying the tree in `tree_root` to
ensure the `TSNode`'s tree cannot be mutated. But that causes the
problem where two calls to `tree_root` return nodes from different
copies of a tree, which do not compare as equal. This has broken at
least one plugin.
Solution:
Make all `TSTree`s on the Lua side always immutable, avoiding the need
to copy the tree in `tree_root`, and make the only mutation point,
`tree_edit`, copy the tree instead.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
TSNode contains a `const TSTree*` and a `const void *id`. The `id`
points to Tree-sitter's internal type `Subtree`, which resides inside
the `TSTree` but may be deallocated if the `TSTree` is mutated (which
is likely why it is `const`).
The Lua method `TSTree:edit()` mutates the tree, which can deallocate
`id`.
See #25254 and #31758.
Solution:
To avoid this, we now make a copy of the tree before pushing its root to
the Lua stack. This also removes the fenv from TSLuaTree, as it was only
used when pushing the tree root to the Lua stack.
We also copy the tree in `node_tree`.
`ts_tree_copy()` just increments a couple of reference counters, so it's
relatively cheap to call.
|
| | |
|
| |
|
|
|
|
| |
Now that we have bumped to tree-sitter 0.25.4, we no longer need to do
this since upstream does it for us when calling the regular parse
method.
|
| |
|
|
|
| |
"Error" in error messages is redundant. Just provide the context, don't
say "Error ...".
|
| |
|
|
|
|
|
|
|
|
|
| |
Apparently after parsing with options in tree-sitter, the options data
persists in the parser object, and thus successive calls to
`ts_parser_parse()` will act like `ts_parser_parse_with_options()`. This
is problematic because `languagetree.lua` makes coroutine-environment
assumptions based on if a nullptr has been returned by the parser
function. This commit makes it so that the parse options state is reset
upon a regular parse (would be nice if this was done upstream).
Fixes #33277
|
| |
|
|
|
| |
Remove the `set_timeout` functions for `TSParser` and instead add a timeout
parameter to the regular parse function. Remove these deprecated tree-sitter
API functions and replace them with the preferred `TSParseOptions` style.
|
| |
|
|
|
|
|
|
|
| |
Problem: Cannot disable individual captures and patterns in treesitter queries.
Solution:
* Expose the corresponding tree-sitter API functions for `TSQuery` object.
* Add documentation for `TSQuery`.
* Return the pattern ID from `get_captures_at_pos()` (and hence `:Inspect!`).
|
| |
|
|
|
|
|
|
|
| |
Problem: No way to check the version of a treesitter parser.
Solution: Add version metadata (ABI 15 parsers only) as well as parser state count and supertype information (ABI 15) in `vim.treesitter.language.inspect()`. Also graduate the `abi_version` field, as this is now the official upstream name.
---------
Co-authored-by: Christian Clason <c.clason@uni-graz.at>
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
Problem:
When running an initial parse, parse() returns an empty table rather
than an actual range. In `languagetree.lua`, we manually check if
a parse was incremental to determine the changed parse region.
Solution:
- Always return a range (in the C side) from parse().
- Simplify the language tree code a bit.
- Logger no longer shows empty ranges on the initial parse.
|
| |
|
|
|
| |
Breaking change: `ts_node_child_containing_descendant()` was removed
Breaking change: tree-sitter 0.25 (HEAD) required
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
**Problem:** Parsing can be slow for large files, and it is a blocking
operation which can be disruptive and annoying.
**Solution:** Provide a function for asynchronous parsing, which accepts
a callback to be run after parsing completes.
Co-authored-by: Lewis Russell <lewis6991@gmail.com>
Co-authored-by: Luuk van Baal <luukvbaal@gmail.com>
Co-authored-by: VanaIgr <vanaigranov@gmail.com>
|
| |
|
|
|
|
| |
Regression from 2a7d0ed6145bf3f8b139c2694563f460f829813a, which removed
header that is only needed if wasmtime support is enabled. Prevent this
from happening again by wrapping the include in a `HAVE_WASMTIME` check.
|
| |
|
| |
Result of `make iwyu` (after some "fixups").
|
| |
|
|
|
|
|
|
|
|
|
| |
**Problem:** Currently node names with non-alphanumeric, non
underscore/hyphen characters (only possible with anonymous nodes) are
not given a proper error message. See tree-sitter issue 3892 for more
details.
**Solution:** Apply a different scanning logic to anonymous nodes to
correctly identify the entire node name (i.e., up until the final double
quote)
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
**Problem:** Tree-sitter 0.24.0 introduced a new symbol type to denote
supertype nodes (`TSSymbolTypeSupertype`). Now, `language.inspect()`
(and the query `omnifunc`) return supertype symbols, but with double
quotes around them.
**Solution:** Mark a symbol as "named" based on it *not* being an
anonymous node, rather than checking that it is a regular node (which a
supertype also is not).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
**Problems:**
- `vim.treesitter.language.inspect()` returns duplicate
symbol names, sometimes up to 6 of one kind in the case of `markdown`
- The list-like `symbols` table can have holes and is thus not even a
valid msgpack table anyway, mentioned in a test
**Solution:** Return symbols as a map, rather than a list, where field
names are the names of the symbol. The boolean value associated with the
field encodes whether or not the symbol is named.
Note that anonymous nodes are surrounded with double quotes (`"`) to
prevent potential collisions with named counterparts that have the same
identifier.
|
| |
|
|
|
|
| |
This commit also marks `child_containing_descendant()` as deprecated
(per upstream's documentation), and uses `child_with_descendant()` in
its place. Minimum required tree-sitter version will now be `0.24`.
|
| |
|
|
|
|
|
|
|
| |
Problem:
Tree cursors can only be efficient when they are re-used.
Short-lived cursors are very slow.
Solution:
Reimplement functions that use short-lived cursors.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: Installing treesitter parser is hard (harder than
climbing to heaven).
Solution: Add optional support for wasm parsers with `wasmtime`.
Notes:
* Needs to be enabled by setting `ENABLE_WASMTIME` for tree-sitter and
Neovim. Build with
`make CMAKE_EXTRA_FLAGS=-DENABLE_WASMTIME=ON
DEPS_CMAKE_FLAGS=-DENABLE_WASMTIME=ON`
* Adds optional Rust (obviously) and C11 dependencies.
* Wasmtime comes with a lot of features that can negatively affect
Neovim performance due to library and symbol table size. Make sure to
build with minimal features and full LTO.
* To reduce re-compilation times, install `sccache` and build with
`RUSTC_WRAPPER=<path/to/sccache> make ...`
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Problem: `has-ancestor?` is O(n²) for the depth of the tree since it iterates over each of the node's ancestors (bottom-up), and each ancestor takes O(n) time.
This happens because tree-sitter's nodes don't store their parent nodes, and the tree is searched (top-down) each time a new parent is requested.
Solution: Make use of new `ts_node_child_containing_descendant()` in tree-sitter v0.22.6 (which is now the minimum required version) to rewrite the `has-ancestor?` predicate in C to become O(n).
For a sample file, decreases the time taken by `has-ancestor?` from 360ms to 6ms.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Instead of painfully messing with timing to determine if queries were
reparsed, we can simply keep a counter next to the call to ts_query_new
Also memoization had a hidden dependency on the garbage collection of
the the key, a hash value which never is kept around in memory. this was
done intentionally as the hash does not capture all relevant state for the
query (external included files) even if actual query objects still
would be reachable in memory. To make the test fully deterministic in
CI, we explicitly control GC.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
- Also address some coverity warnings
Fixes #27942
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
`TSNode:_rawquery()` is complicated, has known issues and the Lua and
C code is awkwardly coupled (see logic with `active`).
Solution:
- Add `TSQueryCursor` and `TSQueryMatch` bindings.
- Replace `TSNode:_rawquery()` with `TSQueryCursor:next_capture()` and `TSQueryCursor:next_match()`
- Do more stuff in Lua
- API for `Query:iter_captures()` and `Query:iter_matches()` remains the same.
- `treesitter.c` no longer contains any logic related to predicates.
- Add `match_limit` option to `iter_matches()`. Default is still 256.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Query patterns can contain quantifiers (e.g. (foo)+ @bar), so a single
capture can map to multiple nodes. The iter_matches API can not handle
this situation because the match table incorrectly maps capture indices
to a single node instead of to an array of nodes.
The match table should be updated to map capture indices to an array of
nodes. However, this is a massively breaking change, so must be done
with a proper deprecation period.
`iter_matches`, `add_predicate` and `add_directive` must opt-in to the
correct behavior for backward compatibility. This is done with a new
"all" option. This option will become the default and removed after the
0.10 release.
Co-authored-by: Christian Clason <c.clason@uni-graz.at>
Co-authored-by: MDeiml <matthias@deiml.net>
Co-authored-by: Gregory Anders <greg@gpanders.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- `TSQuery`: userdata object for parsed query.
- `vim.treesitter.Query`: renamed from `Query`.
- Add a new field `lang`.
- `TSQueryInfo`:
- Move to `vim/treesitter/_meta.lua`, because C code owns it.
- Correct typing for `patterns`, should be a map from `integer`
(pattern_id) to `(integer|string)[][]` (list of predicates or
directives).
- `vim.treesitter.QueryInfo` is added.
- This currently has the same structure as `TSQueryInfo` (exported
from C code).
- Document the fields (see `TSQuery:inspect`).
- Add typing for `vim._ts_parse_query()`.
|
| |
|
| |
It's the "tree-sitter" project, but "treesitter" in our code and docs.
|
| | |
|
| | |
|
| | |
|
