From bd20892bdf15e45c4ea5827250e3aa0c2ace24a0 Mon Sep 17 00:00:00 2001 From: "Justin M. Keyes" Date: Sat, 11 Jun 2016 13:57:02 -0400 Subject: doc: Move spell design discussion. It is worth preserving, but let it live in spell.txt so we can dedicate develop.txt to nvim-specific discussion. --- runtime/doc/develop.txt | 105 ------------------------------------------------ 1 file changed, 105 deletions(-) (limited to 'runtime/doc/develop.txt') diff --git a/runtime/doc/develop.txt b/runtime/doc/develop.txt index 8881845fdd..7ced8dd016 100644 --- a/runtime/doc/develop.txt +++ b/runtime/doc/develop.txt @@ -153,110 +153,5 @@ window View on a buffer. There can be several windows in Vim, fit in the shell. -Spell checking *develop-spell* - -When spell checking was going to be added to Vim a survey was done over the -available spell checking libraries and programs. Unfortunately, the result -was that none of them provided sufficient capabilities to be used as the spell -checking engine in Vim, for various reasons: - -- Missing support for multi-byte encodings. At least UTF-8 must be supported, - so that more than one language can be used in the same file. - Doing on-the-fly conversion is not always possible (would require iconv - support). -- For the programs and libraries: Using them as-is would require installing - them separately from Vim. That's mostly not impossible, but a drawback. -- Performance: A few tests showed that it's possible to check spelling on the - fly (while redrawing), just like syntax highlighting. But the mechanisms - used by other code are much slower. Myspell uses a hashtable, for example. - The affix compression that most spell checkers use makes it slower too. -- For using an external program like aspell a communication mechanism would - have to be setup. That's complicated to do in a portable way (Unix-only - would be relatively simple, but that's not good enough). And performance - will become a problem (lots of process switching involved). -- Missing support for words with non-word characters, such as "Etten-Leur" and - "et al.", would require marking the pieces of them OK, lowering the - reliability. -- Missing support for regions or dialects. Makes it difficult to accept - all English words and highlight non-Canadian words differently. -- Missing support for rare words. Many words are correct but hardly ever used - and could be a misspelled often-used word. -- For making suggestions the speed is less important and requiring to install - another program or library would be acceptable. But the word lists probably - differ, the suggestions may be wrong words. - - -Spelling suggestions *develop-spell-suggestions* - -For making suggestions there are two basic mechanisms: -1. Try changing the bad word a little bit and check for a match with a good - word. Or go through the list of good words, change them a little bit and - check for a match with the bad word. The changes are deleting a character, - inserting a character, swapping two characters, etc. -2. Perform soundfolding on both the bad word and the good words and then find - matches, possibly with a few changes like with the first mechanism. - -The first is good for finding typing mistakes. After experimenting with -hashtables and looking at solutions from other spell checkers the conclusion -was that a trie (a kind of tree structure) is ideal for this. Both for -reducing memory use and being able to try sensible changes. For example, when -inserting a character only characters that lead to good words need to be -tried. Other mechanisms (with hashtables) need to try all possible letters at -every position in the word. Also, a hashtable has the requirement that word -boundaries are identified separately, while a trie does not require this. -That makes the mechanism a lot simpler. - -Soundfolding is useful when someone knows how the words sounds but doesn't -know how it is spelled. For example, the word "dictionary" might be written -as "daktonerie". The number of changes that the first method would need to -try is very big, it's hard to find the good word that way. After soundfolding -the words become "tktnr" and "tkxnry", these differ by only two letters. - -To find words by their soundfolded equivalent (soundalike word) we need a list -of all soundfolded words. A few experiments have been done to find out what -the best method is. Alternatives: -1. Do the sound folding on the fly when looking for suggestions. This means - walking through the trie of good words, soundfolding each word and - checking how different it is from the bad word. This is very efficient for - memory use, but takes a long time. On a fast PC it takes a couple of - seconds for English, which can be acceptable for interactive use. But for - some languages it takes more than ten seconds (e.g., German, Catalan), - which is unacceptable slow. For batch processing (automatic corrections) - it's too slow for all languages. -2. Use a trie for the soundfolded words, so that searching can be done just - like how it works without soundfolding. This requires remembering a list - of good words for each soundfolded word. This makes finding matches very - fast but requires quite a lot of memory, in the order of 1 to 10 Mbyte. - For some languages more than the original word list. -3. Like the second alternative, but reduce the amount of memory by using affix - compression and store only the soundfolded basic word. This is what Aspell - does. Disadvantage is that affixes need to be stripped from the bad word - before soundfolding it, which means that mistakes at the start and/or end - of the word will cause the mechanism to fail. Also, this becomes slow when - the bad word is quite different from the good word. - -The choice made is to use the second mechanism and use a separate file. This -way a user with sufficient memory can get very good suggestions while a user -who is short of memory or just wants the spell checking and no suggestions -doesn't use so much memory. - - -Word frequency - -For sorting suggestions it helps to know which words are common. In theory we -could store a word frequency with the word in the dictionary. However, this -requires storing a count per word. That degrades word tree compression a lot. -And maintaining the word frequency for all languages will be a heavy task. -Also, it would be nice to prefer words that are already in the text. This way -the words that appear in the specific text are preferred for suggestions. - -What has been implemented is to count words that have been seen during -displaying. A hashtable is used to quickly find the word count. The count is -initialized from words listed in COMMON items in the affix file, so that it -also works when starting a new file. - -This isn't ideal, because the longer Vim is running the higher the counts -become. But in practice it is a noticeable improvement over not using the word -count. vim:tw=78:ts=8:ft=help:norl: -- cgit From 23a3e58f02e37ed5c6f8fa1f4758d0ebe1aed8d3 Mon Sep 17 00:00:00 2001 From: "Justin M. Keyes" Date: Mon, 13 Jun 2016 02:11:12 -0400 Subject: doc: msgpack_rpc: De-emphasize the transport protocol. Tighten our jargon, avoid mentioning "msgpack" everywhere we mention the RPC API. Prefer "RPC API" when speaking about the remote API. Though it's conceivable that we may one day support some protocol other than msgpack, that isn't relevant to most of our discussion of the API, including this document. The file name msgpack_rpc.txt is preserved to avoid totally breaking URLs. --- runtime/doc/develop.txt | 37 ++++++++++++++++--------------------- 1 file changed, 16 insertions(+), 21 deletions(-) (limited to 'runtime/doc/develop.txt') diff --git a/runtime/doc/develop.txt b/runtime/doc/develop.txt index 7ced8dd016..4912bb7ce2 100644 --- a/runtime/doc/develop.txt +++ b/runtime/doc/develop.txt @@ -1,23 +1,19 @@ -*develop.txt* For Vim version 7.4. Last change: 2016 Jan 31 +*develop.txt* - VIM REFERENCE MANUAL by Bram Moolenaar + NVIM REFERENCE MANUAL -Development of Vim. *development* - -This text is important for those who want to be involved in further developing -Vim. +Development of Nvim. *development* 1. Design goals |design-goals| 2. Design decisions |design-decisions| -See the file "src/nvim/README.md" for a high-level overview of the source -code. +See src/nvim/README.md for a high-level overview of the source code: + https://github.com/neovim/neovim/blob/master/src/nvim/README.md -Vim is open source software. Everybody is encouraged to contribute to help -improving Vim. For sending patches a context diff "diff -c" is preferred. -Also see http://vim.wikia.com/wiki/How_to_make_and_submit_a_patch. +Nvim is open source software. Everybody is encouraged to contribute. + https://github.com/neovim/neovim/blob/master/CONTRIBUTING.md ============================================================================== 1. Design goals *design-goals* @@ -28,7 +24,7 @@ Note that quite a few items are contradicting. This is intentional. A balance must be found between them. -VIM IS... IMPROVED *design-improved* +NVIM IS... IMPROVED *design-improved* The IMproved bits of Vim should make it a better Vi, without becoming a completely different editor. Extensions are done with a "Vi spirit". @@ -49,7 +45,7 @@ completely different editor. Extensions are done with a "Vi spirit". implement and (3) someone actually implementing it. -VIM IS... MULTI PLATFORM *design-multi-platform* +NVIM IS... MULTI PLATFORM *design-multi-platform* Vim tries to help as many users on as many platforms as possible. - Support many kinds of terminals. The minimal demands are cursor positioning @@ -68,7 +64,7 @@ Vim tries to help as many users on as many platforms as possible. contradicts the previous item, these two must be balanced.] -VIM IS... WELL DOCUMENTED *design-documented* +NVIM IS... WELL DOCUMENTED *design-documented* - A feature that isn't documented is a useless feature. A patch for a new feature must include the documentation. @@ -78,7 +74,7 @@ VIM IS... WELL DOCUMENTED *design-documented* item is easier to find. -VIM IS... HIGH SPEED AND SMALL IN SIZE *design-speed-size* +NVIM IS... HIGH SPEED AND SMALL IN SIZE *design-speed-size* Using Vim must not be a big attack on system resources. Keep it small and fast. @@ -95,7 +91,7 @@ fast. application, but have it work well together with other programs. -VIM IS... MAINTAINABLE *design-maintain* +NVIM IS... MAINTAINABLE *design-maintain* - The source code should not become a mess. It should be reliable code. - Use comments in a useful way! Quoting the function name and argument names @@ -106,7 +102,7 @@ VIM IS... MAINTAINABLE *design-maintain* knowledge spread to other parts of the code. -VIM IS... FLEXIBLE *design-flexible* +NVIM IS... FLEXIBLE *design-flexible* Vim should make it easy for users to work in their preferred styles rather than coercing its users into particular patterns of work. This can be for @@ -117,10 +113,9 @@ be used to adjust Vim to the desire of the user and its environment. NVIM IS... NOT *design-not* -Nvim is not an Operating System; instead it should be composed with other -tools, or hosted as a component. Marvim once said: "Unlike Emacs, Nvim does -not attempt to include everything but the kitchen sink, but some people use it -for plumbing." +Nvim is not an operating system—instead it should be composed with other tools +or hosted as a component. Marvim once said: "Unlike Emacs, Nvim does not +include the kitchen sink... but you can use it for plumbing." ============================================================================== -- cgit From cc6edfe357cfe139c6f08ea645d9ff6aa8cf008c Mon Sep 17 00:00:00 2001 From: "Justin M. Keyes" Date: Tue, 14 Jun 2016 23:12:43 -0400 Subject: doc: remove "nvim-" qualfier from tags It is almost never necessary to qualify documentation as "Nvim specific". The main exception to this is vim_diff.txt. --- runtime/doc/develop.txt | 30 +++++++++++------------------- 1 file changed, 11 insertions(+), 19 deletions(-) (limited to 'runtime/doc/develop.txt') diff --git a/runtime/doc/develop.txt b/runtime/doc/develop.txt index 4912bb7ce2..658de0dce9 100644 --- a/runtime/doc/develop.txt +++ b/runtime/doc/develop.txt @@ -9,12 +9,12 @@ Development of Nvim. *development* 1. Design goals |design-goals| 2. Design decisions |design-decisions| -See src/nvim/README.md for a high-level overview of the source code: - https://github.com/neovim/neovim/blob/master/src/nvim/README.md - Nvim is open source software. Everybody is encouraged to contribute. https://github.com/neovim/neovim/blob/master/CONTRIBUTING.md +See src/nvim/README.md for a high-level overview of the source code: + https://github.com/neovim/neovim/blob/master/src/nvim/README.md + ============================================================================== 1. Design goals *design-goals* @@ -72,6 +72,9 @@ NVIM IS... WELL DOCUMENTED *design-documented* recommended. - Don't make the text unnecessarily long. Less documentation means that an item is easier to find. +- Do not prefix doc-tags with "nvim-". Use |vim_diff.txt| to document + differences from Vim. The {Nvim} annotation is also available + to mark a specific feature. No other distinction is necessary. NVIM IS... HIGH SPEED AND SMALL IN SIZE *design-speed-size* @@ -85,8 +88,6 @@ fast. possible. Useful commands may take longer. - Don't forget that some people use Vim over a slow connection. Minimize the communication overhead. -- Items that add considerably to the size and are not used by many people - should be a feature that can be disabled. - Vim is a component among other components. Don't turn it into a massive application, but have it work well together with other programs. @@ -113,25 +114,14 @@ be used to adjust Vim to the desire of the user and its environment. NVIM IS... NOT *design-not* -Nvim is not an operating system—instead it should be composed with other tools -or hosted as a component. Marvim once said: "Unlike Emacs, Nvim does not +Nvim is not an operating system; instead it should be composed with other +tools or hosted as a component. Marvim once said: "Unlike Emacs, Nvim does not include the kitchen sink... but you can use it for plumbing." ============================================================================== 2. Design decisions *design-decisions* -Folding - -Several forms of folding should be possible for the same buffer. For example, -have one window that shows the text with function bodies folded, another -window that shows a function body. - -Folding is a way to display the text. It should not change the text itself. -Therefore the folding has been implemented as a filter between the text stored -in a buffer (buffer lines) and the text displayed in a window (logical lines). - - Naming the window The word "window" is commonly used for several things: A window on the screen, @@ -147,6 +137,8 @@ window View on a buffer. There can be several windows in Vim, together with the command line, menubar, toolbar, etc. they fit in the shell. - +RPC API +API client +remote plugin vim:tw=78:ts=8:ft=help:norl: -- cgit From 7718f8f24c9e795103b639367a90992dd161bd01 Mon Sep 17 00:00:00 2001 From: "Justin M. Keyes" Date: Wed, 15 Jun 2016 01:20:08 -0400 Subject: doc/provider: cleanup - Move design/impl discussion to develop.txt - Move clipboard notes to provider.txt --- runtime/doc/develop.txt | 47 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-) (limited to 'runtime/doc/develop.txt') diff --git a/runtime/doc/develop.txt b/runtime/doc/develop.txt index 658de0dce9..4013544311 100644 --- a/runtime/doc/develop.txt +++ b/runtime/doc/develop.txt @@ -122,7 +122,7 @@ include the kitchen sink... but you can use it for plumbing." ============================================================================== 2. Design decisions *design-decisions* -Naming the window +Window The word "window" is commonly used for several things: A window on the screen, the xterm window, a window inside Vim to view a buffer. @@ -137,6 +137,51 @@ window View on a buffer. There can be several windows in Vim, together with the command line, menubar, toolbar, etc. they fit in the shell. + +Providers *dev-provider* + +A goal of Nvim is to allow extension of the editor without special knowledge +in the core. But some Vim components are too tightly coupled; in those cases +a "provider" hook is exposed. + +Consider two examples of integration with external systems that are +implemented in Vim and are now decoupled from Nvim core as providers: + +1. In the Vim source code, clipboard logic accounts for more than 1k lines of + C source code (ui.c), to perform two tasks that are now accomplished with + shell commands such as xclip or pbcopy/pbpaste. + +2. Python scripting support: Vim has three files dedicated to embedding the + Python interpreter: if_python.c, if_python3.c and if_py_both.h. Together + these files sum about 9.5k lines of C source code. In contrast, Nvim Python + scripting is performed by an external host process implemented in ~2k lines + of Python. + +Ideally we could implement Python and clipboard integration in pure vimscript +and without touching the C code. But this is infeasible without compromising +backwards compatibility with Vim; that's where providers help. + +The provider framework helps call vimscript from C. It is composed of two +functions in eval.c: + +- eval_call_provider(name, method, arguments): calls provider#(name)#Call + with the method and arguments. +- eval_has_provider(name): Checks if a provider is implemented. Returns true + if the provider#(name)#Call function is implemented. Called by |has()| + (vimscript) to check if features are available. + +The provider#(name)#Call function implements integration with an external +system, because calling shell commands and |rpc| clients are easier to work +with in vimscript. + +For example, the Python provider is implemented by the +autoload/provider/python.vim script; the provider#python#Call function is only +defined if a valid external Python host is found. That works well with the +`has('python')` expression (normally used by Python plugins) because if the +Python host isn't installed then the plugin will "think" it is running in +a Vim compiled without the |+python| feature. + + RPC API API client remote plugin -- cgit From 2dbf0400483be7c92c9a2f389ffa33ec48d8dff0 Mon Sep 17 00:00:00 2001 From: "Justin M. Keyes" Date: Wed, 15 Jun 2016 01:59:28 -0400 Subject: doc: uppercase RPC --- runtime/doc/develop.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'runtime/doc/develop.txt') diff --git a/runtime/doc/develop.txt b/runtime/doc/develop.txt index 4013544311..4c02dba3d6 100644 --- a/runtime/doc/develop.txt +++ b/runtime/doc/develop.txt @@ -171,8 +171,8 @@ functions in eval.c: (vimscript) to check if features are available. The provider#(name)#Call function implements integration with an external -system, because calling shell commands and |rpc| clients are easier to work -with in vimscript. +system, because shell commands and |RPC| clients are easier to work with in +vimscript. For example, the Python provider is implemented by the autoload/provider/python.vim script; the provider#python#Call function is only -- cgit From 742787fe9e9016971edf0eefc7cd9a1d19f73008 Mon Sep 17 00:00:00 2001 From: "Justin M. Keyes" Date: Wed, 15 Jun 2016 21:33:47 -0400 Subject: doc: consolidate nvim.txt --- runtime/doc/develop.txt | 2 ++ 1 file changed, 2 insertions(+) (limited to 'runtime/doc/develop.txt') diff --git a/runtime/doc/develop.txt b/runtime/doc/develop.txt index 4c02dba3d6..1b77d87aac 100644 --- a/runtime/doc/develop.txt +++ b/runtime/doc/develop.txt @@ -75,6 +75,8 @@ NVIM IS... WELL DOCUMENTED *design-documented* - Do not prefix doc-tags with "nvim-". Use |vim_diff.txt| to document differences from Vim. The {Nvim} annotation is also available to mark a specific feature. No other distinction is necessary. +- If a feature is removed, delete its doc entry and move its tag to + |vim_diff.txt|. NVIM IS... HIGH SPEED AND SMALL IN SIZE *design-speed-size* -- cgit