aboutsummaryrefslogtreecommitdiff
path: root/runtime/lua/vim/treesitter/languagetree.lua
Commit message (Collapse)AuthorAge
* refactor(treesitter): migrate to ts parser callback API #33141Riley Bruins2025-03-29
| | | | | Remove the `set_timeout` functions for `TSParser` and instead add a timeout parameter to the regular parse function. Remove these deprecated tree-sitter API functions and replace them with the preferred `TSParseOptions` style.
* refactor(treesitter): simplify injection retrieval #33104Riley Bruins2025-03-28
| | | | | | | Simplify the logic for retrieving the injection ranges for the language tree. The trees are now also sorted by starting position, regardless of whether they are part of a combined injection or not. This would be helpful if ranges are ever to be stored in an interval tree or other kind of sorted tree structure.
* refactor(treesitter): simplify parsing coroutine logicRiley Bruins2025-02-23
| | | | | | | | | Lua coroutines can yield across non-coroutine function boundaries, meaning that we don't need to wrap each helper function in a coroutine and resume it within `_parse()`. If we just have them yield when appropriate, this will be caught by the top level `_parse()` coroutine, and resuming the `_parse()` will resume from the position in the helper function where we yielded last.
* perf(treesitter): don't block when finding injection rangesRiley Bruins2025-02-21
| | | | | | | | | | | **Problem:** Currently, parsing is asynchronous, but it involves a (sometimes lengthy) step which finds all injection ranges for a tree by iterating over that language's injection queries. This causes edits in large files to be extremely slow, and also causes a long stutter during the initial parse of a large file. **Solution:** Break up the injection query iteration over multiple event loop iterations.
* perf(treesitter): only search for injections within the parse rangeRiley Bruins2025-02-21
| | | | Co-authored-by: Jaehwang Jung <tomtomjhj@gmail.com>
* fix(treesitter): separately track the number of valid regionsRiley Bruins2025-02-17
| | | | | | We need to add a separate variable to keep track of this information, since we cannot read the length of the valid regions table itself, since it has holes.
* fix(treesitter): detect trees with outdated regions in `is_valid()`Riley Bruins2025-02-11
|
* feat(treesitter): allow LanguageTree:is_valid() to accept a rangeRiley Bruins2025-02-02
| | | | | | When given, only that range will be checked for validity rather than the entire tree. This is used in the highlighter to save CPU cycles since we only need to parse a certain region at a time anyway.
* refactor(treesitter): use coroutines for resuming _parse() logicRiley Bruins2025-02-02
| | | | | | This means that all work previously done by a `_parse()` iteration will be kept in future iterations. This prevents it from running indefinitely in some cases where the file is very large and there are 2+ injections.
* refactor(treesitter): always return valid range from parse() #32273Riley Bruins2025-02-02
| | | | | | | | | | | Problem: When running an initial parse, parse() returns an empty table rather than an actual range. In `languagetree.lua`, we manually check if a parse was incremental to determine the changed parse region. Solution: - Always return a range (in the C side) from parse(). - Simplify the language tree code a bit. - Logger no longer shows empty ranges on the initial parse.
* refactor(treesitter): drop `LanguageTree._has_regions` #32274Riley Bruins2025-02-02
| | | | | | | | | | | | | | | | This simplifies some logic in `languagetree.lua`, removing the need for `_has_regions`, and removing side effects in `:included_regions()`. Before: - Edit is made which sets `_regions = nil` - Upon the next call to `included_regions()` (usually right after we marked `_regions` as `nil` due to an `_iter_regions()` call), if `_regions` is nil, we repopulate the table (as long as the tree actually has regions) After: - Edit is made which resets `_regions` if it exists - `included_regions()` no longer needs to perform this logic itself, and also no longer needs to read a `_has_regions` variable
* fix(treesitter): nil access when running string parser asyncRiley Bruins2025-02-01
|
* docs: miscdundargoc2025-01-30
| | | | | | | | | Co-authored-by: Dustin S. <dstackmasta27@gmail.com> Co-authored-by: Ferenc Fejes <fejes@inf.elte.hu> Co-authored-by: Maria José Solano <majosolano99@gmail.com> Co-authored-by: Yochem van Rosmalen <git@yochem.nl> Co-authored-by: brianhuster <phambinhanctb2004@gmail.com> Co-authored-by: zeertzjq <zeertzjq@outlook.com>
* fix(treesitter): stop async parsing if buffer is invalidnotomo2025-01-29
| | | | | Problem: Error occurs if delete buffer in the middle of parsing. Solution: Check if buffer is valid in parsing.
* fix(treesitter): empty queries can disable injections (#31748)Riley Bruins2025-01-28
| | | | | | | | | | | | **Problem:** Currently, if users want to efficiently disable injections, they have to delete the injection query files at their runtime path. This is because we only check for existence of the files before running the query over the entire buffer. **Solution:** Check for existence of query files, *and* that those files actually have captures. This will allow users to just comment out existing queries (or better yet, just add their own injection query to `~/.config/nvim` which contains only comments) to disable running the query over the entire buffer (a potentially slow operation)
* fix: resolve all remaining LuaLS diagnosticsLewis Russell2025-01-27
|
* docs(treesitter): expose LanguageTree:parent() #32108Jaehwang Jung2025-01-20
| | | | | Plugins may want to climb up the LanguageTree. Also add missing type annotations for other methods.
* fix(treesitter): clean up parsing queueJaehwang Jung2025-01-19
|
* feat(treesitter): async parsingRiley Bruins2025-01-12
| | | | | | | | | | | | **Problem:** Parsing can be slow for large files, and it is a blocking operation which can be disruptive and annoying. **Solution:** Provide a function for asynchronous parsing, which accepts a callback to be run after parsing completes. Co-authored-by: Lewis Russell <lewis6991@gmail.com> Co-authored-by: Luuk van Baal <luukvbaal@gmail.com> Co-authored-by: VanaIgr <vanaigranov@gmail.com>
* refactor(treesitter): simplify condition #31889Riley Bruins2025-01-06
|
* fix(treesitter): remove redundant on_bytes callback #31041luukvbaal2024-11-16
| | | | | | | Problem: Treesitter highlighter implements an on_bytes callback that just re-marks a buffer range for redraw. The edit that prompted the callback will already have done that. Solution: Remove redundant on_bytes callback from the treesitter highlighter module.
* feat(treesitter)!: use return values in `language.add()`Christian Clason2024-09-29
| | | | | | | | | Problem: No clear way to check whether parsers are available for a given language. Solution: Make `language.add()` return `true` if a parser was successfully added and `nil` otherwise. Use explicit `assert` instead of relying on thrown errors.
* feat(treesitter)!: default to correct behavior for quantified captures (#30193)Gregory Anders2024-09-01
| | | | | | | | | | | | For context, see https://github.com/neovim/neovim/pull/24738. Before that PR, Nvim did not correctly handle captures with quantifiers. That PR made the correct behavior opt-in to minimize breaking changes, with the intention that the correct behavior would eventually become the default. Users can still opt-in to the old (incorrect) behavior for now, but this option will eventually be removed completely. BREAKING CHANGE: Any plugin which uses `Query:iter_matches()` must update their call sites to expect an array of nodes in the `match` table, rather than a single node.
* docs(treesitter): generate inline docs for `Range`sYi Ming2024-08-06
| | | | | | | | docs(treesitter): in-place parameter description docs(treesitter): remove internal type names docs(treesitter): add missing private annotation
* feat(treesitter): add node_for_range functionRiley Bruins2024-07-29
| | | | | | This is identical to `named_node_for_range` except that it includes anonymous nodes. This maintains consistency in the API because we already have `descendant_for_range` and `named_descendant_for_range`.
* refactor(lua): improve type annotationsLewis Russell2024-06-11
|
* feat: remove deprecated featuresdundargoc2024-05-16
| | | | | | | | | | | | | | | | | | | Remove following functions: - vim.lsp.util.extract_completion_items - vim.lsp.util.get_progress_messages - vim.lsp.util.parse_snippet() - vim.lsp.util.text_document_completion_list_to_complete_items - LanguageTree:for_each_child - health#report_error - health#report_info - health#report_ok - health#report_start - health#report_warn - vim.health.report_error - vim.health.report_info - vim.health.report_ok - vim.health.report_start - vim.health.report_warn
* docs: newsJustin M. Keyes2024-05-15
| | | | Set dev_xx.txt help files to use "flow" layout.
* fix(treesitter): enforce lowercase language names (#28546)Christian Clason2024-04-28
| | | | | | | | | | | * fix(treesitter): enforce lowercase language names Problem: On case-insensitive file systems (e.g., macOS), `has_parser` will return `true` for uppercase aliases, which will then try to inject the uppercase language unsuccessfully. Solution: Enforce and assume parser names to be lowercase when resolving language names.
* fix(treesitter): use tree range instead of tree root node rangealtermo2024-04-10
|
* feat(treesitter): add `@injection.filename`Christian Clason2024-04-02
| | | | | | | | | | | | | Problem: Injecting languages for file redirects (e.g., in bash) is not possible. Solution: Add `@injection.filename` capture that is piped through `vim.filetype.match({ filename = node_text })`; the resulting filetype (if not `nil`) is then resolved as a language (either directly or through the list maintained via `vim.treesitter.language.register()`). Note: `@injection.filename` is a non-standard capture introduced by Helix; having two editors implement it makes it likely to be upstreamed.
* refactor(lua): type annotationsLewis Russell2024-03-16
|
* fix(treesitter): highlight injections properlyLewis Russell2024-03-14
| | | | | `on_line_impl` doesn't highlight single lines, so using pattern indexes to offset priority doesn't work.
* docs: support inline markdownLewis Russell2024-03-09
| | | | | | - Tags are now created with `[tag]()` - References are now created with `[tag]` - Code spans are no longer wrapped
* refactor(types): more fixes (2)Lewis Russell2024-03-06
|
* docs: improve/add documentation of Lua typesLewis Russell2024-03-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Added `@inlinedoc` so single use Lua types can be inlined into the functions docs. E.g. ```lua --- @class myopts --- @inlinedoc --- --- Documentation for some field --- @field somefield integer --- @param opts myOpts function foo(opts) end ``` Will be rendered as ``` foo(opts) Parameters: - {opts} (table) Object with the fields: - somefield (integer) Documentation for some field ``` - Marked many classes with with `@nodoc` or `(private)`. We can eventually introduce these when we want to.
* feat(docs): replace lua2dox.luaLewis Russell2024-02-27
| | | | | | | | | | | | | | | | | | | | | | | | | | Problem: The documentation flow (`gen_vimdoc.py`) has several issues: - it's not very versatile - depends on doxygen - doesn't work well with Lua code as it requires an awkward filter script to convert it into pseudo-C. - The intermediate XML files and filters makes it too much like a rube goldberg machine. Solution: Re-implement the flow using Lua, LPEG and treesitter. - `gen_vimdoc.py` is now replaced with `gen_vimdoc.lua` and replicates a portion of the logic. - `lua2dox.lua` is gone! - No more XML files. - Doxygen is now longer used and instead we now use: - LPEG for comment parsing (see `scripts/luacats_grammar.lua` and `scripts/cdoc_grammar.lua`). - LPEG for C parsing (see `scripts/cdoc_parser.lua`) - Lua patterns for Lua parsing (see `scripts/luacats_parser.lua`). - Treesitter for Markdown parsing (see `scripts/text_utils.lua`). - The generated `runtime/doc/*.mpack` files have been removed. - `scripts/gen_eval_files.lua` now instead uses `scripts/cdoc_parser.lua` directly. - Text wrapping is implemented in `scripts/text_utils.lua` and appears to produce more consistent results (the main contributer to the diff of this change).
* fix(treesitter): correctly handle query quantifiers (#24738)Thomas Vigouroux2024-02-16
| | | | | | | | | | | | | | | | | | | Query patterns can contain quantifiers (e.g. (foo)+ @bar), so a single capture can map to multiple nodes. The iter_matches API can not handle this situation because the match table incorrectly maps capture indices to a single node instead of to an array of nodes. The match table should be updated to map capture indices to an array of nodes. However, this is a massively breaking change, so must be done with a proper deprecation period. `iter_matches`, `add_predicate` and `add_directive` must opt-in to the correct behavior for backward compatibility. This is done with a new "all" option. This option will become the default and removed after the 0.10 release. Co-authored-by: Christian Clason <c.clason@uni-graz.at> Co-authored-by: MDeiml <matthias@deiml.net> Co-authored-by: Gregory Anders <greg@gpanders.com>
* fix(treesitter): validate language alias for injectionsChristian Clason2024-01-18
| | | | | | | | Problem: Parsed language annotations can be random garbage so `nvim_get_runtime_file` throws an error. Solution: Validate that `alias` is a valid language name before trying to find a parser for it.
* fix(treesitter): outdated highlight due to tree with outdated regionJaehwang Jung2023-12-24
| | | | | | | | | | | | | Problem: A region managed by an injected parser may shrink after re-running the injection query. If the updated region goes out of the range to be parsed, then the corresponding tree will remain outdated, possibly retaining the nodes that shouldn't exist anymore. This results in outdated highlights. Solution: Re-parse an invalid tree if its region intersects the range to be parsed.
* fix(treesitter): don't invalidate parser when discovering injectionsDmytro Soltys2023-11-27
| | | | | | | | | When parsing with a range, languagetree looks up injections and adds them if needed. This explicitly invalidates parser, making `is_valid` report `false` both when including and excluding children. This is an attempt to describe desired behaviour of `is_valid` in tests, with what ended up being a single line change to satisfy them.
* fix(languagetree): don't treat unparsed nodes as occupying full rangeL Lllvvuu2023-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | This is incorrect in the following scenario: 1. The language tree is Lua > Vim > Lua. 2. An edit simultaneously wipes out the `_regions` of all nodes, while taking the Vim injection off-screen. 3. The Vim injection is not re-parsed, so the child Lua `_regions` is still `nil`. 4. The child Lua is assumed, incorrectly, to occupy the whole document. 5. This causes the injections to be parsed again, resulting in Lua > Vim > Lua > Vim. 6. Now, by the same process, Vim ends up with its range assumed over the whole document. Now the parse is broken and results in broken highlighting and poor performance. It should be fine to instead treat an unparsed node as occupying nothing (i.e. effectively non-existent). Since, either: - The parent was just parsed, hence defining `_regions` - The parent was not just parsed, in which case this node doesn't need to be parsed either. Also, the name `has_regions` is confusing; it seems to simply mean the opposite of "root" or "full_document". However, this PR does not touch it.
* feat(lua): add vim.func._memoizeLewis Russell2023-09-20
| | | | | | | | | | | | | | | Memoizes a function, using a custom function to hash the arguments. Private for now until: - There are other places in the codebase that could benefit from this (e.g. LSP), but might require other changes to accommodate. - Invalidation of the cache needs to be controllable. Using weak tables is an acceptable invalidation policy, but it shouldn't be the only one. - I don't think the story around `hash_fn` is completely thought out. We may be able to have a good default hash_fn by hashing each argument, so basically a better 'concat'.
* fix(treesitter): _trees may not be list-likeJaehwang Jung2023-09-17
| | | | | | | | | | | Problem: With incremental injection parsing, injected languages' parsers parse only the relevant regions and stores the result in _trees with the index of the corresponding region. Therefore, there can be holes in _trees. Solution: * Use generic table functions where appropriate. * Fix type annotations and docs.
* fix(treesitter): properly combine injection.combined regionsJaehwang Jung2023-09-16
| | | | | | | | | Problem: It doesn't make much sense to flatten each region (= list of ranges). This coincidentally worked for region with a single range. Solution: Custom function for combining regions.
* fix(languagetree): apply `resolve_lang` to `metadata['injection.language']`L Lllvvuu2023-09-16
| | | | | | | | | | | | | `resolve_lang` is applied to `@injection.language` when it's supplied as a capture: https://github.com/neovim/neovim/blob/f5953edbac14febce9d4f8a3c35bdec1eae26fbe/runtime/lua/vim/treesitter/languagetree.lua#L766-L768 If we want to support `metadata['injection.language']` (as per #22518 and [tree-sitter upstream](https://tree-sitter.github.io/tree-sitter/syntax-highlighting#language-injection)) then the behavior should be consistent. Fixes: nvim-treesitter/nvim-treesitter#4918
* docs: replace <pre> with ``` (#25136)Gregory Anders2023-09-14
|
* refactor(treesitter): deprecate for_each_child #25118LW2023-09-14
| | | | | | | The name for_each_child is misleading and caused bugs. After #25111, #25115, there are no more usages of `for_each_child` in Nvim. In the future if we want to restore this functionality we can consider a generalized vim.traverse(node, key, visitor) function.
* fix(treesitter): remove more double recursionLewis Russell2023-09-12
| | | | Do not call `for_each_child` in functions that are already recursive.
* fix(languagetree): remove double recursion in LanguageTree:parseL Lllvvuu2023-09-12
| | | | | | | | | | | | | | | | `LanguageTree:parse` is recursive, and calls `LanguageTree:for_each_child`, which is also recursive. That means that, starting from the third level (child of child of root), nodes will be parsed twice. Which then means that if the tree is N layers deep, there will be ~2^N parses even if the branching factor is 1. Now, why was the tree deepening with each character inserted? And why did this only regress in #24647? These are mysteries for another time. Fixes: #25104
generated by cgit (git 2.34.1) at 2025-08-30 23:56:43 +0000