From 2ca076e45fb3f1c08f6a1a374834df0701b8d778 Mon Sep 17 00:00:00 2001
From: Lewis Russell <lewis6991@gmail.com>
Date: Thu, 10 Aug 2023 14:21:56 +0100
Subject: feat(treesitter)!: incremental injection parsing

Problem:

Treesitter highlighting is slow for large files with lots of injections.

Solution:

Only parse injections we are going to render during a redraw cycle.

---

- `LanguageTree:parse()` will no longer parse injections by default and
  now requires an explicit range argument to be passed.

- `TSHighlighter` now parses injections incrementally during on_win
  callbacks for the line range being rendered.

- Plugins which require certain injections to be parsed must run
  `parser:parse({ start_row, end_row })` before using the tree.
---
 runtime/doc/news.txt       |  7 +++++++
 runtime/doc/treesitter.txt | 31 ++++++++++++++++++-------------
 2 files changed, 25 insertions(+), 13 deletions(-)

(limited to 'runtime/doc')

diff --git a/runtime/doc/news.txt b/runtime/doc/news.txt
index a1981d5b7f..3aacf1e8ee 100644
--- a/runtime/doc/news.txt
+++ b/runtime/doc/news.txt
@@ -61,6 +61,10 @@ The following changes may require adaptations in user config or plugins.
     spaces (but paths themselves may contain spaces now).
   • |'directory'| will no longer remove a `>` at the start of the option.
 
+• |LanguageTree:parse()| will no longer parse injections by default and
+  now requires an explicit range argument to be passed. If injections are
+  required, provide an explicit range via `parser:parse({ start_row, end_row })`.
+
 ==============================================================================
 NEW FEATURES                                                    *news-features*
 
@@ -69,6 +73,9 @@ The following new APIs and features were added.
 • Performance:
   • 'diffopt' "linematch" scoring algorithm now favours larger and less groups
     https://github.com/neovim/neovim/pull/23611
+  • Treesitter highlighting now parses injections incrementally during
+    screen redraws only for the line range being rendered. This significantly
+    improves performance in large files with many injections.
 
 • |vim.iter()| provides a generic iterator interface for tables and Lua
   iterators |for-in|.
diff --git a/runtime/doc/treesitter.txt b/runtime/doc/treesitter.txt
index 13c0bd024a..f3e697807f 100644
--- a/runtime/doc/treesitter.txt
+++ b/runtime/doc/treesitter.txt
@@ -1019,13 +1019,6 @@ set({lang}, {query_name}, {text})                 *vim.treesitter.query.set()*
       • {text}        (string) Query text (unparsed).
 
 
-==============================================================================
-Lua module: vim.treesitter.highlighter            *lua-treesitter-highlighter*
-
-TSHighlighter:destroy()                              *TSHighlighter:destroy()*
-    Removes all internal references to the highlighter.
-
-
 ==============================================================================
 Lua module: vim.treesitter.languagetree          *lua-treesitter-languagetree*
 
@@ -1053,7 +1046,7 @@ Whenever you need to access the current syntax tree, parse the buffer:
 
 >lua
 
-     local tree = parser:parse()
+     local tree = parser:parse({ start_row, end_row })
 
 <
 
@@ -1112,7 +1105,7 @@ LanguageTree:included_regions()              *LanguageTree:included_regions()*
     Gets the set of included regions
 
     Return: ~
-        integer[][]
+        Range6[][]
 
 LanguageTree:invalidate({reload})                  *LanguageTree:invalidate()*
     Invalidates this parser and all its children
@@ -1155,10 +1148,22 @@ LanguageTree:named_node_for_range({range}, {opts})
     Return: ~
         |TSNode| | nil Found node
 
-LanguageTree:parse()                                    *LanguageTree:parse()*
-    Parses all defined regions using a treesitter parser for the language this
-    tree represents. This will run the injection query for this language to
-    determine if any child languages should be created.
+LanguageTree:parse({range})                             *LanguageTree:parse()*
+    Recursively parse all regions in the language tree using
+    |treesitter-parsers| for the corresponding languages and run injection
+    queries on the parsed trees to determine whether child trees should be
+    created and parsed.
+
+    Any region with empty range (`{}`, typically only the root tree) is always
+    parsed; otherwise (typically injections) only if it intersects {range} (or
+    if {range} is `true`).
+
+    Parameters: ~
+      • {range}  boolean|Range|nil: Parse this range in the parser's source.
+                 Set to `true` to run a complete parse of the source (Note:
+                 Can be slow!) Set to `false|nil` to only parse regions with
+                 empty ranges (typically only the root tree without
+                 injections).
 
     Return: ~
         TSTree[]
-- 
cgit