aboutsummaryrefslogtreecommitdiff
path: root/runtime/lua/vim/treesitter/languagetree.lua
Commit message (Collapse)AuthorAge
* fix(treesitter): don't invalidate parser when discovering injectionsDmytro Soltys2023-11-27
| | | | | | | | | When parsing with a range, languagetree looks up injections and adds them if needed. This explicitly invalidates parser, making `is_valid` report `false` both when including and excluding children. This is an attempt to describe desired behaviour of `is_valid` in tests, with what ended up being a single line change to satisfy them.
* fix(languagetree): don't treat unparsed nodes as occupying full rangeL Lllvvuu2023-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | This is incorrect in the following scenario: 1. The language tree is Lua > Vim > Lua. 2. An edit simultaneously wipes out the `_regions` of all nodes, while taking the Vim injection off-screen. 3. The Vim injection is not re-parsed, so the child Lua `_regions` is still `nil`. 4. The child Lua is assumed, incorrectly, to occupy the whole document. 5. This causes the injections to be parsed again, resulting in Lua > Vim > Lua > Vim. 6. Now, by the same process, Vim ends up with its range assumed over the whole document. Now the parse is broken and results in broken highlighting and poor performance. It should be fine to instead treat an unparsed node as occupying nothing (i.e. effectively non-existent). Since, either: - The parent was just parsed, hence defining `_regions` - The parent was not just parsed, in which case this node doesn't need to be parsed either. Also, the name `has_regions` is confusing; it seems to simply mean the opposite of "root" or "full_document". However, this PR does not touch it.
* feat(lua): add vim.func._memoizeLewis Russell2023-09-20
| | | | | | | | | | | | | | | Memoizes a function, using a custom function to hash the arguments. Private for now until: - There are other places in the codebase that could benefit from this (e.g. LSP), but might require other changes to accommodate. - Invalidation of the cache needs to be controllable. Using weak tables is an acceptable invalidation policy, but it shouldn't be the only one. - I don't think the story around `hash_fn` is completely thought out. We may be able to have a good default hash_fn by hashing each argument, so basically a better 'concat'.
* fix(treesitter): _trees may not be list-likeJaehwang Jung2023-09-17
| | | | | | | | | | | Problem: With incremental injection parsing, injected languages' parsers parse only the relevant regions and stores the result in _trees with the index of the corresponding region. Therefore, there can be holes in _trees. Solution: * Use generic table functions where appropriate. * Fix type annotations and docs.
* fix(treesitter): properly combine injection.combined regionsJaehwang Jung2023-09-16
| | | | | | | | | Problem: It doesn't make much sense to flatten each region (= list of ranges). This coincidentally worked for region with a single range. Solution: Custom function for combining regions.
* fix(languagetree): apply `resolve_lang` to `metadata['injection.language']`L Lllvvuu2023-09-16
| | | | | | | | | | | | | `resolve_lang` is applied to `@injection.language` when it's supplied as a capture: https://github.com/neovim/neovim/blob/f5953edbac14febce9d4f8a3c35bdec1eae26fbe/runtime/lua/vim/treesitter/languagetree.lua#L766-L768 If we want to support `metadata['injection.language']` (as per #22518 and [tree-sitter upstream](https://tree-sitter.github.io/tree-sitter/syntax-highlighting#language-injection)) then the behavior should be consistent. Fixes: nvim-treesitter/nvim-treesitter#4918
* docs: replace <pre> with ``` (#25136)Gregory Anders2023-09-14
|
* refactor(treesitter): deprecate for_each_child #25118LW2023-09-14
| | | | | | | The name for_each_child is misleading and caused bugs. After #25111, #25115, there are no more usages of `for_each_child` in Nvim. In the future if we want to restore this functionality we can consider a generalized vim.traverse(node, key, visitor) function.
* fix(treesitter): remove more double recursionLewis Russell2023-09-12
| | | | Do not call `for_each_child` in functions that are already recursive.
* fix(languagetree): remove double recursion in LanguageTree:parseL Lllvvuu2023-09-12
| | | | | | | | | | | | | | | | `LanguageTree:parse` is recursive, and calls `LanguageTree:for_each_child`, which is also recursive. That means that, starting from the third level (child of child of root), nodes will be parsed twice. Which then means that if the tree is N layers deep, there will be ~2^N parses even if the branching factor is 1. Now, why was the tree deepening with each character inserted? And why did this only regress in #24647? These are mysteries for another time. Fixes: #25104
* feat(treesitter): add 'injection.self' and 'injection.parent'Amaan Qureshi2023-08-24
| | | | Co-authored-by: ObserverOfTime <chronobserver@disroot.org>
* fix(treesitter)!: remove deprecated legacy injection formatChristian Clason2023-08-14
|
* fix(treesitter): logger memory leakLewis Russell2023-08-13
|
* feat(treesitter)!: incremental injection parsingLewis Russell2023-08-12
| | | | | | | | | | | | | | | | | | | | | Problem: Treesitter highlighting is slow for large files with lots of injections. Solution: Only parse injections we are going to render during a redraw cycle. --- - `LanguageTree:parse()` will no longer parse injections by default and now requires an explicit range argument to be passed. - `TSHighlighter` now parses injections incrementally during on_win callbacks for the line range being rendered. - Plugins which require certain injections to be parsed must run `parser:parse({ start_row, end_row })` before using the tree.
* feat(treesitter): add injection language fallback (#24659)Christian Clason2023-08-11
| | | | | | | | | | | | * feat(treesitter): add injection language fallback Problem: injection languages are often specified via aliases (e.g., filetype or in upper case), requiring custom directives. Solution: include lookup logic (try as parser name, then filetype, then lowercase) in LanguageTree itself and remove `#inject-language` directive. Co-authored-by: Lewis Russell <me@lewisr.dev>
* fix(treesitter): make sure injections don't return empty ranges (#24595)Lewis Russell2023-08-07
| | | | | | | | | | | | | | | | | | | | When an injection has not set include children, make sure not to add the injection if no ranges are determined. This could happen when there is an injection with a child that has the same range as itself. e.g. consider this Makefile snippet ```make foo: $(VAR) ``` Line 2 has an injection for bash and a make variable reference. If include-children isn't set (default), then there is no range on line 2 to inject since the variable reference needs to be excluded. This caused the language tree to return an empty range, which the parser now interprets to mean the full buffer. This caused makefiles to have completely broken highlighting.
* docs(lua): more improvements (#24387)Lewis Russell2023-07-18
| | | | | | | | | | | | | | | | | * docs(lua): teach lua2dox how to table * docs(lua): teach gen_vimdoc.py about local functions No more need to mark local functions with @private * docs(lua): mention @nodoc and @meta in dev-lua-doc * fixup! Co-authored-by: Justin M. Keyes <justinkz@gmail.com> --------- Co-authored-by: Justin M. Keyes <justinkz@gmail.com>
* fix(treesitter): handle empty region when logging (#24173)Jaehwang Jung2023-06-27
|
* feat(lua): rename vim.loop -> vim.uv (#22846)Lewis Russell2023-06-03
|
* fix: function was renamed (#23772)Matthieu Coudron2023-05-26
|
* feat(treesitter): improved logging (#23638)Lewis Russell2023-05-17
| | | | | | - Add bindings to Treesitter ts_parser_set_logger and ts_parser_logger - Add logfile with path STDPATH('log')/treesitter.c - Rework existing LanguageTree loggin to use logfile - Begin implementing log levels for vim.g.__ts_debug
* docs: small fixesdundargoc2023-05-13
| | | | | | | | | Co-authored-by: Christian Clason <c.clason@uni-graz.at> Co-authored-by: Gregory Anders <greg@gpanders.com> Co-authored-by: HiPhish <hiphish@posteo.de> Co-authored-by: Julio B <julio.bacel@gmail.com> Co-authored-by: T727 <74924917+T-727@users.noreply.github.com> Co-authored-by: camoz <camoz@users.noreply.github.com> Co-authored-by: champignoom <66909116+champignoom@users.noreply.github.com>
* fix(treesitter): foldexpr tweaksLewis Russell2023-05-01
| | | Some small general fixes found working on developing async parsing.
* fix(treesitter): redraw added/removed injections properly (#23287)Lewis Russell2023-04-30
| | | | | | When injections are added or removed make sure to: - invoke 'changedtree' callbacks for when new trees are added. - invoke 'changedtree' callbacks for when trees are invalidated - redraw regions when languagetree children are removed
* fix(treesitter): do not track ranges of the root tree (#22912)Lewis Russell2023-04-06
| | | Fixes #22911
* refactor: rename local API alias from a to apiLewis Russell2023-04-05
| | | | | | | | Problem: Codebase inconsistently binds vim.api onto a or api. Solution: Use api everywhere. a as an identifier is too short to have at the module level.
* refactor(treesitter): delegate region calculation to treesitter (#22576)Lewis Russell2023-04-04
|
* feat: add `vim.treesitter.language.get_filetypes()` (#22643)Lewis Russell2023-03-30
|
* feat(treesitter)!: deprecate top level indexes to modules (#22761)Lewis Russell2023-03-24
| | | | | | | | | | | | | | | | | The following top level Treesitter functions have been moved: - vim.treesitter.inspect_language() -> vim.treesitter.language.inspect() - vim.treesitter.get_query_files() -> vim.treesitter.query.get_files() - vim.treesitter.set_query() -> vim.treesitter.query.set() - vim.treesitter.query.set_query() -> vim.treesitter.query.set() - vim.treesitter.get_query() -> vim.treesitter.query.get() - vim.treesitter.query.get_query() -> vim.treesitter.query.get() - vim.treesitter.parse_query() -> vim.treesitter.query.parse() - vim.treesitter.query.parse_query() -> vim.treesitter.query.parse() - vim.treesitter.add_predicate() -> vim.treesitter.query.add_predicate() - vim.treesitter.add_directive() -> vim.treesitter.query.add_directive() - vim.treesitter.list_predicates() -> vim.treesitter.query.list_predicates() - vim.treesitter.list_directives() -> vim.treesitter.query.list_directives() - vim.treesitter.query.get_range() -> vim.treesitter.get_range() - vim.treesitter.query.get_node_text() -> vim.treesitter.get_node_text()
* fix(treesitter): annotationsLewis Russell2023-03-23
| | | | | | - Begin using `@package` in place of `@private` for functions that are accessed internally but outside their defined class. - Rename Node -> TSP.Node
* Merge pull request #22613 from lewis6991/feat/tsqueryutilLewis Russell2023-03-11
|\
| * refactor(treesitter): add Range type aliase for Range4|Range6Lewis Russell2023-03-11
| |
| * feat(treesitter)!: consolidate query util functionsLewis Russell2023-03-10
| | | | | | | | | | | | - And address more type errors. - Removed the `concat` option from `get_node_text` since it was applied inconsistently and made typing awkward.
* | feat!(treesitter): do not return changes from LanguageTree:parse() Lewis Russell2023-03-10
|/ | | | | | | | | Never return the changes an only notify them using the `on_changedtree` callback. It is not guaranteed for a plugin that it'll be the first one to call `tree:parse()` and thus get the changes. Closes #19915
* refactor(treesitter): use byte ranges from treesitter (#22589)Lewis Russell2023-03-09
|
* Revert "refactor(treesitter): delegate region calculation to treesitter" ↵Lewis Russell2023-03-08
| | | | | | | (#22575) Revert "refactor(treesitter): delegate region calculation to treesitter (#22553)" This reverts commit 276b647fdba07bf1762d8dd371c4b655b8a418df.
* refactor(treesitter): delegate region calculation to treesitter (#22553)Lewis Russell2023-03-08
|
* feat(treesitter): use upstream format for injection queriesLewis Russell2023-03-08
|
* docs: module-level docstrings (@defgroup) #22498Justin M. Keyes2023-03-05
| | | | | | | | | | | | Problem: gen_vimdoc.py / lua2dox.lua does not support @defgroup or \defgroup except for "api-foo" modules. Solution: Modify `gen_vimdoc.py` to look for section names based on `helptag_fmt`. TODO: - Support @module ? https://github.com/LuaLS/lua-language-server/wiki/Annotations#module
* docs: fix vim.treesitter tagsJustin M. Keyes2023-03-03
| | | | | | | | | Problem: Help tags like vim.treesitter.language.add() are confusing because `vim.treesitter.language` is (thankfully) not a user-facing module. Solution: Ignore the "fstem" when generating "treesitter" tags.
* fix(treesitter): ipairs -> pairsLewis Russell2023-02-27
| | | Fixes: https://github.com/nvim-treesitter/nvim-treesitter/issues/4349
* feat(treesitter)!: remove silent option from language.add()Lewis Russell2023-02-24
| | | Simply use `pcall` if you want to silence an error.
* fix(treesitter): fixup language invalidation (#22381)Lewis Russell2023-02-23
|
* perf(treesitter): smarter languagetree invalidationLewis Russell2023-02-23
| | | | | | | | | | | | | | Problem: Treesitter injections are slow because all injected trees are invalidated on every change. Solution: Implement smarter invalidation to avoid reparsing injected regions. - In on_bytes, try and update self._regions as best we can. This PR just offsets any regions after the change. - Add valid flags for each region in self._regions. - Call on_bytes recursively for all children. - We still need to run the query every time for the top level tree. I don't know how to avoid this. However, if the new injection ranges don't change, then we re-use the old trees and avoid reparsing children. This should result in roughly a 2-3x reduction in tree parsing when the comment injections are enabled.
* docs(treesitter): fix parse errorsChristian Clason2023-02-23
|
* feat(treesitter): add filetype -> lang APILewis Russell2023-02-21
| | | | | | | | | | | | | | | | | | Problem: vim.treesitter does not know how to map a specific filetype to a parser. This creates problems since in a few places (including in vim.treesitter itself), the filetype is incorrectly used in place of lang. Solution: Add an API to enable this: - Add vim.treesitter.language.add() as a replacement for vim.treesitter.language.require_language(). - Optional arguments are now passed via an opts table. - Also takes a filetype (or list of filetypes) so we can keep track of what filetypes are associated with which langs. - Deprecated vim.treesitter.language.require_language(). - Add vim.treesitter.language.get_lang() which returns the associated lang for a given filetype. - Add vim.treesitter.language.register() to associate filetypes to a lang without loading the parser.
* feat(treesitter): add metadata option for get_node_textfigsoda2023-02-04
|
* feat(treesitter): allow capture text to be transformedfigsoda2023-02-04
| | | | Co-authored-by: Lewis Russell <lewis6991@gmail.com>
* fix(treesitter): fix most diagnosticsLewis Russell2023-02-04
|
* feat(treesitter): add vim.treesitter.show_tree() (#21322)Gregory Anders2022-12-08
| | | | | | | Add a "show_tree" function to view a textual representation of the nodes in a language tree in a window. Moving the cursor in the window highlights the corresponding text in the source buffer, and moving the cursor in the source buffer highlights the corresponding nodes in the window.
generated by cgit (git 2.34.1) at 2025-08-13 19:37:04 +0000