feat!: rewrite TOhtml in lua

Co-authored-by: wookayin <wookayin@gmail.com>
Co-authored-by: clason <c.clason@uni-graz.at>
Co-authored-by: Lewis Russell <me@lewisr.dev>

4 files changed, 38 insertions, 435 deletions

diff --git a/runtime/doc/lua.txt b/runtime/doc/lua.txt
index 2b269f7d9c..5f6f6fb149 100644
--- a/runtime/doc/lua.txt
+++ b/runtime/doc/lua.txt
@@ -4363,4 +4363,35 @@ vim.text.hexencode({str})                               *vim.text.hexencode()*
         (`string`) Hex encoded string
 
 
+==============================================================================
+Lua module: tohtml                                                *vim.tohtml*
+
+
+:TOhtml {file}                                                       *:TOhtml*
+Converts the buffer shown in the current window to HTML, opens the generated
+HTML in a new split window, and saves its contents to {file}. If {file} is not
+given, a temporary file (created by |tempname()|) is used.
+
+
+tohtml.tohtml({winid}, {opt})                         *tohtml.tohtml.tohtml()*
+    Converts the buffer shown in the window {winid} to HTML and returns the
+    output as a list of string.
+
+    Parameters: ~
+      • {winid}  (`integer?`) Window to convert (defaults to current window)
+      • {opt}    (`table?`) Optional parameters.
+                 • title (string): Title tag to set in the generated HTML code
+                   (defaults to buffer name)
+                 • number_lines (boolean): Show line numbers (defaults to
+                   `false`)
+                 • font (string|string[]): Fonts to use (defaults to
+                   `guifont`)
+                 • width (integer) Width used for items which are either right
+                   aligned or repeat a character infinitely (defaults to
+                   'textwidth' if non-zero or window width otherwise)
+
+    Return: ~
+        (`string[]`)
+
+
  vim:tw=78:ts=8:sw=4:sts=4:et:ft=help:norl:
diff --git a/runtime/doc/news.txt b/runtime/doc/news.txt
index 6895254a42..50beb79adf 100644
--- a/runtime/doc/news.txt
+++ b/runtime/doc/news.txt
@@ -134,6 +134,9 @@ The following changes may require adaptations in user config or plugins.
   If necessary, the respective capability can be
   removed when calling |vim.lsp.protocol.make_client_capabilities()|.
 
+• |:TOhtml| has been rewritten in Lua to support Neovim-specific decorations,
+  and many options have been removed.
+
 ==============================================================================
 BREAKING CHANGES IN HEAD                                    *news-breaking-dev*
 
diff --git a/runtime/doc/syntax.txt b/runtime/doc/syntax.txt
index 0e7c38b38d..c02752a2b7 100644
--- a/runtime/doc/syntax.txt
+++ b/runtime/doc/syntax.txt
@@ -341,443 +341,11 @@ Upon loading a file, Vim finds the relevant syntax file as follows:
 	syntax.
 
 ==============================================================================
-4. Conversion to HTML				*2html.vim* *convert-to-HTML*
+4. Conversion to HTML				*convert-to-HTML* *2html.vim*
 
-2html is not a syntax file itself, but a script that converts the current
-window into HTML. Vim opens a new window in which it builds the HTML file.
+The old to html converter has ben replaced by a Lua version and the
+documentation has been moved to |:TOhtml|.
 
-After you save the resulting file, you can view it with any browser. The
-colors should be exactly the same as you see them in Vim.  With
-|g:html_line_ids| you can jump to specific lines by adding (for example) #L123
-or #123 to the end of the URL in your browser's address bar. And with
-|g:html_dynamic_folds| enabled, you can show or hide the text that is folded
-in Vim.
-
-You are not supposed to set the 'filetype' or 'syntax' option to "2html"!
-Source the script to convert the current file: >
-
-	:runtime! syntax/2html.vim
-<
-Many variables affect the output of 2html.vim; see below. Any of the on/off
-options listed below can be enabled or disabled by setting them explicitly to
-the desired value, or restored to their default by removing the variable using
-|:unlet|.
-
-Remarks:
-- Some truly ancient browsers may not show the background colors.
-- From most browsers you can also print the file (in color)!
-
-Here is an example how to run the script over all .c and .h files from a
-Unix shell: >
-   for f in *.[ch]; do gvim -f +"syn on" +"run! syntax/2html.vim" +"wq" +"q" $f; done
-<
-					*g:html_start_line* *g:html_end_line*
-To restrict the conversion to a range of lines, use a range with the |:TOhtml|
-command below, or set "g:html_start_line" and "g:html_end_line" to the first
-and last line to be converted.  Example, using the last set Visual area: >
-
-	:let g:html_start_line = line("'<")
-	:let g:html_end_line = line("'>")
-	:runtime! syntax/2html.vim
-<
-							*:TOhtml*
-:[range]TOhtml		The ":TOhtml" command is defined in a standard plugin.
-			This command will source |2html.vim| for you. When a
-			range is given, this command sets |g:html_start_line|
-			and |g:html_end_line| to the start and end of the
-			range, respectively. Default range is the entire
-			buffer.
-
-			If the current window is part of a |diff|, unless
-			|g:html_diff_one_file| is set, :TOhtml will convert
-			all windows which are part of the diff in the current
-			tab and place them side-by-side in a <table> element
-			in the generated HTML. With |g:html_line_ids| you can
-			jump to lines in specific windows with (for example)
-			#W1L42 for line 42 in the first diffed window, or
-			#W3L87 for line 87 in the third.
-
-			Examples: >
-
-	:10,40TOhtml " convert lines 10-40 to html
-	:'<,'>TOhtml " convert current/last visual selection
-	:TOhtml      " convert entire buffer
-<
-							*g:html_diff_one_file*
-Default: 0.
-When 0, and using |:TOhtml| all windows involved in a |diff| in the current tab
-page are converted to HTML and placed side-by-side in a <table> element. When
-1, only the current buffer is converted.
-Example: >
-
-	let g:html_diff_one_file = 1
-<
-							 *g:html_whole_filler*
-Default: 0.
-When 0, if |g:html_diff_one_file| is 1, a sequence of more than 3 filler lines
-is displayed as three lines with the middle line mentioning the total number
-of inserted lines.
-When 1, always display all inserted lines as if |g:html_diff_one_file| were
-not set.
->
-    :let g:html_whole_filler = 1
-<
-				     *TOhtml-performance* *g:html_no_progress*
-Default: 0.
-When 0, display a progress bar in the statusline for each major step in the
-2html.vim conversion process.
-When 1, do not display the progress bar. This offers a minor speed improvement
-but you won't have any idea how much longer the conversion might take; for big
-files it can take a long time!
-Example: >
-
-	let g:html_no_progress = 1
-<
-You can obtain better performance improvements by also instructing Vim to not
-run interactively, so that too much time is not taken to redraw as the script
-moves through the buffer, switches windows, and the like: >
-
-  vim -E -s -c "let g:html_no_progress=1" -c "syntax on" -c "set ft=c" -c "runtime syntax/2html.vim" -cwqa myfile.c
-<
-Note that the -s flag prevents loading your vimrc and any plugins, so you
-need to explicitly source/enable anything that will affect the HTML
-conversion. See |-E| and |-s-ex| for details. It is probably best to create a
-script to replace all the -c commands and use it with the -u flag instead of
-specifying each command separately.
-
-				    *hl-TOhtmlProgress* *TOhtml-progress-color*
-When displayed, the progress bar will show colored boxes along the statusline
-as the HTML conversion proceeds. By default, the background color as the
-current "DiffDelete" highlight group is used. If "DiffDelete" and "StatusLine"
-have the same background color, TOhtml will automatically adjust the color to
-differ. If you do not like the automatically selected colors, you can define
-your own highlight colors for the progress bar. Example: >
-
-	hi TOhtmlProgress guifg=#c0ffee ctermbg=7
-<
-							 *g:html_number_lines*
-Default: Current 'number' setting.
-When 0, buffer text is displayed in the generated HTML without line numbering.
-When 1, a column of line numbers is added to the generated HTML with the same
-highlighting as the line number column in Vim (|hl-LineNr|).
-Force line numbers even if 'number' is not set: >
-   :let g:html_number_lines = 1
-Force to omit the line numbers: >
-   :let g:html_number_lines = 0
-Go back to the default to use 'number' by deleting the variable: >
-   :unlet g:html_number_lines
-<
-							*g:html_line_ids*
-Default: 1 if |g:html_number_lines| is set, 0 otherwise.
-When 1, adds an HTML id attribute to each line number, or to an empty <span>
-inserted for that purpose if no line numbers are shown. This ID attribute
-takes the form of L123 for single-buffer HTML pages, or W2L123 for diff-view
-pages, and is used to jump to a specific line (in a specific window of a diff
-view). Javascript is inserted to open any closed dynamic folds
-(|g:html_dynamic_folds|) containing the specified line before jumping. The
-javascript also allows omitting the window ID in the url, and the leading L.
-For example: >
-
-	page.html#L123	jumps to line 123 in a single-buffer file
-	page.html#123	does the same
-
-	diff.html#W1L42	jumps to line 42 in the first window in a diff
-	diff.html#42	does the same
-<
-							      *g:html_use_css*
-Default: 1.
-When 1, generate valid HTML 5 markup with CSS styling, supported in all modern
-browsers and many old browsers.
-When 0, generate <font> tags and similar outdated markup. This is not
-recommended but it may work better in really old browsers, email clients,
-forum posts, and similar situations where basic CSS support is unavailable.
-Example: >
-   :let g:html_use_css = 0
-<
-						       *g:html_ignore_conceal*
-Default: 0.
-When 0, concealed text is removed from the HTML and replaced with a character
-from |:syn-cchar| or 'listchars' as appropriate, depending on the current
-value of 'conceallevel'.
-When 1, include all text from the buffer in the generated HTML, even if it is
-|conceal|ed.
-
-Either of the following commands will ensure that all text in the buffer is
-included in the generated HTML (unless it is folded): >
-   :let g:html_ignore_conceal = 1
-   :setl conceallevel=0
-<
-						       *g:html_ignore_folding*
-Default: 0.
-When 0, text in a closed fold is replaced by the text shown for the fold in
-Vim (|fold-foldtext|). See |g:html_dynamic_folds| if you also want to allow
-the user to expand the fold as in Vim to see the text inside.
-When 1, include all text from the buffer in the generated HTML; whether the
-text is in a fold has no impact at all. |g:html_dynamic_folds| has no effect.
-
-Either of these commands will ensure that all text in the buffer is included
-in the generated HTML (unless it is concealed): >
-   zR
-   :let g:html_ignore_folding = 1
-<
-							*g:html_dynamic_folds*
-Default: 0.
-When 0, text in a closed fold is not included at all in the generated HTML.
-When 1, generate javascript to open a fold and show the text within, just like
-in Vim.
-
-Setting this variable to 1 causes 2html.vim to always use CSS for styling,
-regardless of what |g:html_use_css| is set to.
-
-This variable is ignored when |g:html_ignore_folding| is set.
->
-   :let g:html_dynamic_folds = 1
-<
-							*g:html_no_foldcolumn*
-Default: 0.
-When 0, if |g:html_dynamic_folds| is 1, generate a column of text similar to
-Vim's foldcolumn (|fold-foldcolumn|) the user can click on to toggle folds
-open or closed. The minimum width of the generated text column is the current
-'foldcolumn' setting.
-When 1, do not generate this column; instead, hovering the mouse cursor over
-folded text will open the fold as if |g:html_hover_unfold| were set.
->
-   :let g:html_no_foldcolumn = 1
-<
-				*TOhtml-uncopyable-text* *g:html_prevent_copy*
-Default: Empty string.
-This option prevents certain regions of the generated HTML from being copied,
-when you select all text in document rendered in a browser and copy it. Useful
-for allowing users to copy-paste only the source text even if a fold column or
-line numbers are shown in the generated content. Specify regions to be
-affected in this way as follows:
-	f:	fold column
-	n:	line numbers (also within fold text)
-	t:	fold text
-	d:	diff filler
-
-Example, to make the fold column and line numbers uncopyable: >
-	:let g:html_prevent_copy = "fn"
-<
-The method used to prevent copying in the generated page depends on the value
-of |g:html_use_input_for_pc|.
-
-						    *g:html_use_input_for_pc*
-Default: "none"
-If |g:html_prevent_copy| is non-empty, then:
-
-When "all", read-only <input> elements are used in place of normal text for
-uncopyable regions. In some browsers, especially older browsers, after
-selecting an entire page and copying the selection, the <input> tags are not
-pasted with the page text. If |g:html_no_invalid| is 0, the <input> tags have
-invalid type; this works in more browsers, but the page will not validate.
-Note: This method does NOT work in recent versions of Chrome and equivalent
-browsers; the <input> tags get pasted with the text.
-
-When "fallback" (default value), the same <input> elements are generated for
-older browsers, but newer browsers (detected by CSS feature query) hide the
-<input> elements and instead use generated content in an ::before pseudoelement
-to display the uncopyable text. This method should work with the largest
-number of browsers, both old and new.
-
-When "none", the <input> elements are not generated at all. Only the
-generated-content method is used. This means that old browsers, notably
-Internet Explorer, will either copy the text intended not to be copyable, or
-the non-copyable text may not appear at all. However, this is the most
-standards-based method, and there will be much less markup.
-
-							   *g:html_no_invalid*
-Default: 0.
-When 0, if |g:html_prevent_copy| is non-empty and |g:html_use_input_for_pc| is
-not "none", an invalid attribute is intentionally inserted into the <input>
-element for the uncopyable areas. This prevents pasting the <input> elements
-in some applications. Specifically, some versions of Microsoft Word will not
-paste the <input> elements if they contain this invalid attribute. When 1, no
-invalid markup is inserted, and the generated page should validate. However,
-<input> elements may be pasted into some applications and can be difficult to
-remove afterward.
-
-							 *g:html_hover_unfold*
-Default: 0.
-When 0, the only way to open a fold generated by 2html.vim with
-|g:html_dynamic_folds| set, is to click on the generated fold column.
-When 1, use CSS 2.0 to allow the user to open a fold by moving the mouse
-cursor over the displayed fold text. This is useful to allow users with
-disabled javascript to view the folded text.
-
-Note that old browsers (notably Internet Explorer 6) will not support this
-feature.  Browser-specific markup for IE6 is included to fall back to the
-normal CSS1 styling so that the folds show up correctly for this browser, but
-they will not be openable without a foldcolumn.
->
-   :let g:html_hover_unfold = 1
-<
-							      *g:html_id_expr*
-Default: ""
-Dynamic folding and jumping to line IDs rely on unique IDs within the document
-to work. If generated HTML is copied into a larger document, these IDs are no
-longer guaranteed to be unique. Set g:html_id_expr to an expression Vim can
-evaluate to get a unique string to append to each ID used in a given document,
-so that the full IDs will be unique even when combined with other content in a
-larger HTML document. Example, to append _ and the buffer number to each ID: >
-
-	:let g:html_id_expr = '"_" .. bufnr("%")'
-<
-To append a string "_mystring" to the end of each ID: >
-
-	:let g:html_id_expr = '"_mystring"'
-<
-Note: When converting a diff view to HTML, the expression will only be
-evaluated for the first window in the diff, and the result used for all the
-windows.
-
-					  *TOhtml-wrap-text* *g:html_pre_wrap*
-Default: Current 'wrap' setting.
-When 0, if |g:html_no_pre| is 0 or unset, the text in the generated HTML does
-not wrap at the edge of the browser window.
-When 1, if |g:html_use_css| is 1, the CSS 2.0 "white-space:pre-wrap" value is
-used, causing the text to wrap at whitespace at the edge of the browser
-window.
-Explicitly enable text wrapping: >
-   :let g:html_pre_wrap = 1
-Explicitly disable wrapping: >
-   :let g:html_pre_wrap = 0
-Go back to default, determine wrapping from 'wrap' setting: >
-   :unlet g:html_pre_wrap
-<
-							       *g:html_no_pre*
-Default: 0.
-When 0, buffer text in the generated HTML is surrounded by <pre>...</pre>
-tags. Series of whitespace is shown as in Vim without special markup, and tab
-characters can be included literally (see |g:html_expand_tabs|).
-When 1 (not recommended), the <pre> tags are omitted, and a plain <div> is
-used instead. Whitespace is replaced by a series of &nbsp; character
-references, and <br> is used to end each line. This is another way to allow
-text in the generated HTML is wrap (see |g:html_pre_wrap|) which also works in
-old browsers, but may cause noticeable differences between Vim's display and
-the rendered page generated by 2html.vim.
->
-   :let g:html_no_pre = 1
-<
-							       *g:html_no_doc*
-Default: 0.
-When 1 it doesn't generate a full HTML document with a DOCTYPE, <head>,
-<body>, etc. If |g:html_use_css| is enabled (the default) you'll have to
-define the CSS manually. The |g:html_dynamic_folds| and |g:html_line_ids|
-settings (off by default) also insert some JavaScript.
-
-
-							     *g:html_no_links*
-Default: 0.
-Don't generate <a> tags for text that looks like an URL.
-
-							  *g:html_no_modeline*
-Default: 0.
-Don't generate a modeline disabling folding.
-
-							  *g:html_expand_tabs*
-Default: 0 if 'tabstop' is 8, 'expandtab' is 0, 'vartabstop' is not in use,
-	       and no fold column or line numbers occur in the generated HTML;
-	 1 otherwise.
-When 1, <Tab> characters in the buffer text are replaced with an appropriate
-number of space characters, or &nbsp; references if |g:html_no_pre| is 1.
-When 0, if |g:html_no_pre| is 0 or unset, <Tab> characters in the buffer text
-are included as-is in the generated HTML. This is useful for when you want to
-allow copy and paste from a browser without losing the actual whitespace in
-the source document. Note that this can easily break text alignment and
-indentation in the HTML, unless set by default.
-
-Force |2html.vim| to keep <Tab> characters: >
-   :let g:html_expand_tabs = 0
-<
-Force tabs to be expanded: >
-   :let g:html_expand_tabs = 1
-<
-				    *TOhtml-encoding-detect* *TOhtml-encoding*
-It is highly recommended to set your desired encoding with
-|g:html_use_encoding| for any content which will be placed on a web server.
-
-If you do not specify an encoding, |2html.vim| uses the preferred IANA name
-for the current value of 'fileencoding' if set, or 'encoding' if not.
-'encoding' is always used for certain 'buftype' values. 'fileencoding' will be
-set to match the chosen document encoding.
-
-Automatic detection works for the encodings mentioned specifically by name in
-|encoding-names|, but TOhtml will only automatically use those encodings with
-wide browser support. However, you can override this to support specific
-encodings that may not be automatically detected by default (see options
-below). See https://www.iana.org/assignments/character-sets for the IANA names.
-
-Note: By default all Unicode encodings are converted to UTF-8 with no BOM in
-the generated HTML, as recommended by W3C:
-
-	https://www.w3.org/International/questions/qa-choosing-encodings
-	https://www.w3.org/International/questions/qa-byte-order-mark
-
-							 *g:html_use_encoding*
-Default: none, uses IANA name for current 'fileencoding' as above.
-To overrule all automatic charset detection, set g:html_use_encoding to the
-name of the charset to be used. It is recommended to set this variable to
-something widely supported, like UTF-8, for anything you will be hosting on a
-webserver: >
-   :let g:html_use_encoding = "UTF-8"
-You can also use this option to omit the line that specifies the charset
-entirely, by setting g:html_use_encoding to an empty string (NOT recommended): >
-   :let g:html_use_encoding = ""
-To go back to the automatic mechanism, delete the |g:html_use_encoding|
-variable: >
-   :unlet g:html_use_encoding
-<
-						    *g:html_encoding_override*
-Default: none, autoload/tohtml.vim contains default conversions for encodings
-		mentioned by name at |encoding-names|.
-This option allows |2html.vim| to detect the correct 'fileencoding' when you
-specify an encoding with |g:html_use_encoding| which is not in the default
-list of conversions.
-
-This is a dictionary of charset-encoding pairs that will replace existing
-pairs automatically detected by TOhtml, or supplement with new pairs.
-
-Detect the HTML charset "windows-1252" as the encoding "8bit-cp1252": >
-   :let g:html_encoding_override = {'windows-1252': '8bit-cp1252'}
-<
-						     *g:html_charset_override*
-Default: none, autoload/tohtml.vim contains default conversions for encodings
-		mentioned by name at |encoding-names| and which have wide
-		browser support.
-This option allows |2html.vim| to detect the HTML charset for any
-'fileencoding' or 'encoding' which is not detected automatically. You can also
-use it to override specific existing encoding-charset pairs. For example,
-TOhtml will by default use UTF-8 for all Unicode/UCS encodings. To use UTF-16
-and UTF-32 instead, use: >
-   :let g:html_charset_override = {'ucs-4': 'UTF-32', 'utf-16': 'UTF-16'}
-
-Note that documents encoded in either UTF-32 or UTF-16 have known
-compatibility problems with some major browsers.
-
-								 *g:html_font*
-Default: "monospace"
-You can specify the font or fonts used in the converted document using
-g:html_font. If this option is set to a string, then the value will be
-surrounded with single quotes. If this option is set to a list then each list
-item is surrounded by single quotes and the list is joined with commas. Either
-way, "monospace" is added as the fallback generic family name and the entire
-result used as the font family (using CSS) or font face (if not using CSS).
-Examples: >
-
-   " font-family: 'Consolas', monospace;
-   :let g:html_font = "Consolas"
-
-   " font-family: 'DejaVu Sans Mono', 'Consolas', monospace;
-   :let g:html_font = ["DejaVu Sans Mono", "Consolas"]
-<
-			*convert-to-XML* *convert-to-XHTML* *g:html_use_xhtml*
-Default: 0.
-When 0, generate standard HTML 4.01 (strict when possible).
-When 1, generate XHTML 1.0 instead (XML compliant HTML).
->
-    :let g:html_use_xhtml = 1
-<
 ==============================================================================
 5. Syntax file remarks					*:syn-file-remarks*
 
diff --git a/runtime/doc/vim_diff.txt b/runtime/doc/vim_diff.txt
index 1d5d62c737..b0caf9fdaf 100644
--- a/runtime/doc/vim_diff.txt
+++ b/runtime/doc/vim_diff.txt
@@ -635,6 +635,7 @@ Commands:
   :lcscope
   :scscope
   :Vimuntar
+  The old `:TOhtml`, replaced by a Lua version (contains many differences)
 
 Compile-time features:
   Emacs tags support