FIXES:

2abdb89522fc fix failed release: regenerate the *.mpack file

This release represents ~2700 commits since v0.3.4, the previous
non-maintenance release.  Besides the highlights listed below, this
release features vast improvements to documentation, internal subsystems
and test/CI infrastructure, and 700+ patches merged from Vim.

FEATURES:

New API functions:
  nvim_create_buf: create various kinds of buffers
  nvim_get_context, nvim_load_context
    8e6b0a73c91b #10619 API: Context: save/restore/inspect editor state
  nvim_input_mouse: perform mouse actions
  nvim_open_win: create floating windows (and external, for supporting UIs)
  nvim_paste: paste text at cursor
  nvim_put: put text at cursor
  nvim_select_popupmenu_item: perform popupmenu actions
  nvim_set_keymap: create/delete mappings
  nvim_set_vvar: set v: variables
  nvim_ui_pum_set_height
  nvim_ui_try_resize_grid
  nvim_win_close: close windows
  nvim_win_get_config: get window configuration
  nvim_win_set_config: reconfigure windows

New UI events:
  redraw.grid_destroy
  redraw.hl_group_set
    8a3f8589a3ed #10504 UI/highlight: expose builtin highlight groups using hl_group_set event
  redraw.msg_clear
  redraw.msg_history_show
  redraw.msg_ruler
  redraw.msg_set_pos
  redraw.msg_show
  redraw.msg_showcmd
  redraw.msg_showmode
  redraw.win_close
  redraw.win_external_pos
  redraw.win_float_pos
  redraw.win_hide
  redraw.win_pos

API
f5c56f03bb9e #9170 API/Lua: nvim_buf_attach: support Lua callback
82d48c0dab0f #9896 API: emit nvim_error_event on failed async request
b9ad12e6c2fa #9992 UI/nvim_ui_attach(): add `override` option
3d1ed7c95901 #9993 UI/ext_messages: learn more message kinds
8ed54bbec3b0 #9547 proper multiline error message for rpcrequest, API wrappers

Lua
This release introduces "Nvim-Lua standard library". See ":help lua-stdlib".
89d7e24891c2 #9463 Lua stdlib: vim.inspect, string functions
8e941c59ec23 #9740 Lua: generate documentation from docstrings
1cbe0145695b #9301 lua/stdlib: Introduce vim.shared
c83926cd0aa5 #10123 Lua: introduce vim.loop (expose libuv event-loop)
81e1dbca99c1 #10120 Lua: vim.schedule(cb)
1f54f68732b4 #10688 Lua: minimal UTF-16 support needed for LSP
6fb0020df4eb #10513 Lua encoding support
    c0993ed3433e Lua: support getting UTF-32 and UTF-16 sizes of replaced text
    b0e26199ec02 Lua: add {old_byte_size} to on_lines buffer change event

UI:
- The Nvim 0.3.4 UI protocol introduced line-based updates instead of
  legacy char-based updates. Nvim 0.4 continues to evolve the UI
  protocol. See ":help ui". Legacy UI clients are supported. See
  ":help api-contract".
9a1675b06539 #6619 Floating windows
  - Can be (re)positioned, anchored, external.
  - Are real windows showing real buffers. No shortcuts, hacks, or compromises.
  - Support all features and API of normal windows, plus more.
6427894d8911 #8455 Multigrid: "windows drawn on separate grids"
  - Windows are logically isolated internally.
  - Windows are sent to UIs as distinct objects, so that UIs can control
    layout instead of being stuck with the classic TUI layout.
  - Per-window font-size, dimenions, line-spacing.
  - Compositor: Internal subsystem for composing grids.
3855204f5860 #6917 UIEnter, UILeave
788bcbba2465 #9923 ui: ":syn blend=", 'winblend'
7cf7c0a0b8e9 #9575 ui: 'redrawdebug' option for flexible debugging of redrawing
5c836d2ef8b6 #9607 wildoptions=pum (enabled by default)
37f8df882463 #9571 UI: 'pumblend' option for semi-transparent popupmenu
c403a95a5297 #9446 Visual: highlight char-at-cursor
  - Traditionally Vim's visual selection does "reverse mode", which
    perhaps conflicts with the non-blinking block cursor. But
    'guicursor' defaults to a vertical bar for selection=exclusive, and
    this confuses users who expect to see the text highlighted.

:terminal
fc27dc98d74f #8550 autocmds: TermEnter, TermLeave
d13803f64fc5 #9810 keymap, terminal: more keycodes
3b56f59532ac #9535 :terminal : Fix F1-F4 key codes
2d4a37ebab35 #10370 :ls : show "R", "F" for terminal-jobs
fd0fd752c872 #9966 terminal: swap priority of terminal, editor highlights
7bb858c39cac #9494 libvterm 0.1

TUI
3afb397407af syntax, TUI: support "strikethrough"
ccbcd390d42d #9408 TUI: "title stacking" unconditionally
298608f88c46 #9509 TUI: detect background color, set bg=dark/light
42f492ac9905 #9097 TUI: handle Smulx extension capability (extended underline)
424ddd01f588 #10205 TUI: support rgba background detection
9b4383261a3d #9601 TUI: italics in tmux, Terminal.app
f6fb370b1bcc #9793 keymap: support more (keypad) keycodes
3340e08becbf #9423 TUI: Konsole DECSCUSR fixup

:checkhealth
d0fd66ba82c4 health/provider.vim: check curl HTTPS support
c38862aceabe #10490 checkhealth: try yarn if npm is missing
43356a43d00f #9929 health: check if tmux enabled true colors
ec5a4d862d71 #9548 checkhealth: validate locale

providers (clipboard, python, etc.):
96be8a2c4d63 #10161 Allow reloading providers (useful for UIs/clients)
db3c797c6b3c #9487 provider: improve error message if provider is missing

Various:
36762a00a801 #9295 signs: support multiple columns
801fe799ff35 #10382 eval: wait() (wait for any condition)
9df3a676e7f2 #10400 MsgArea highlight; message grid
a9bea8c1047f #10790 keymap: allow modifiers to multibyte chars, like <m-ä>
25e0a449bb66 #10878 #4448 paste: redesign (10x+ faster pasting; extensible vim.paste Lua hook)
ef5037e7f6e1 #9706 autocmd: introduce "++once" feature
175398f21645 #9616 add CompleteChanged autocmd
7fcf2f926fc6 #9717 TextYankPost: add v:event["inclusive"]
3a699a790c14 #8364 termdebug.vim plugin
ca1ce590257c #9709 performance: use os_copy to create backups
ed0e96cd28f8 man.vim: set 'linebreak'
70f6939fd4b5 #9564 events: add "Signal" event
f89d0d8230f3 #9568 inccommand: auto-disable if folding is slow

FIXES:

41bb68b8e8d0 #10584 process_stop: uv: do not close stdin first/explicitly
e50aa2a6c654 #10117 normal: Don't exit CTRL-O mode after processing K_EVENT
95fa71c6d2b4 #9504 :recover : Fix crash on non-existent *.swp
5a836d4767b7 #9507 screen: don't unconditionally clear messages on window scroll
149dcbf2c762 #10021 channel: refactor events, prevent recursive invocation of events
d19ff73b39a9 #10107 Fix multiple c_CTRL-D showing statusline
b65a7b7f6692 #10103 Fix wildmode=list,full and display+=msgsep interaction
0be6d3c86fe5 #9634 fsync: Ignore ENOTSUP. Fix writing to SMB.
b247c6fd2255 #10025 kbtree: pointer UB and unitialized value fixes
018e0d5a19c3 #9643 API/buffer-updates: always detach on buf-reload
400ee59247ea #9961 API: fix cursor position when lines are added
769f44e918c6 #9911 win/defaults: Use "…/nvim-data/site" in 'runtimepath'
83d571653bdc #9911 spellfile.vim: store files in stdpath('data')
8dbf23181add #9887 RPC: conform message-id type to msgpack-RPC spec
5f996e36d102 #9894 options: properly reset directories on 'autochdir'
4c4a57015687 #9807 various CursorMoved fixes
943bedfc86ab #9853 event-loop: do not set CA_COMMAND_BUSY
9d207fd87617 #9693 dictwatcheradd(): support b:changedtick
2d50bf349883 #9789 mac: fix locale detection
c5631338b16b #9754 :mksession : restore tab-local working directories
092e7e6c6058 #9703 #9703 executable(): return false if user is not owner
11a481f711ee #9686 env var fixes/improvements
8e54847fdf3c #9666 #7920 os/env: Fix completion of multibyte env var names
519382646be3 #10468 Fix is_executable_in_path() on Windows
8eaa452073a1 #9516 win: exepath(), executable() fixes
f55c1e4233a4 #10544 reltimefloat(): allow negative result
b08dc3ec195f #10561 win: jobstart(), system(): $PATHEXT-resolve exe
7cc2b723d43c #10392 TextYankPost: spurious/too-early dispatch during delete
6e01ed6a4c85 OpenBSD: stop jobs/processes properly
58dd5fcc01ed #10522 jobstop(): close channel before process_stop()
83632022f84e #10959 improved resize behavior (all UIs)
c6eb1f42bec0 #10830 API: fix nvim_command_output buffer overflow
cbfd18c85acd #10763 startup: handle 'guicursor' after user config
b8f2436febcc #10915 jobwait(): fix race if job exits quickly
2fafed6bb8aa #10765 clipboard: handle/avoid SIGTERM with previous owner
8aca932aa0ac #9954 clipboard: setreg("*") with clipboard=unnamed
3f10c5b5338f #9480 performance: clipboard/macOS: assume that pbcopy works
48efafc81c84 #10398 screen: disable redrawing inside VimResized
5e4b93a38f1e #10389 API/Lua: make nvim_execute_lua use native lua floats, not special tables
8c6f5b7f9268 #9934 Spurious quote mark in command line when typing <C-R>
a8a38f346548 Lua 5.2/5.3 compat

:terminal
47b7b471fa0f #10700 :terminal : update buffer when switching tabpage
5225c1ec302f #9605 terminal: Fix potential invalid local 'scrollback'
894f6bee54e8 #8325 :terminal : set topline based on window height
8171e96b9640 #9551 Improve :terminal resize
d928b036dc2b #9856 :stopinsert should leave terminal-mode
3f712185058f #9926 :terminal : fix: Using `:stopinsert` while in normal mode
5020daa6e5ce #9883 ui/terminal: make terminal state redraw like any other state

TUI:
9f19e8d29dce #9443 TUI: Do not disable BCE for builtin terminfos
a4076e5dcf66 #9474 win/TUI: fix text overrides line numbers
533d4a36ec03 #9645 TUI: do not resize host-terminal on startup
b51e5d8b8dd2 #9688 tui_tk_ti_getstr: handle weird value
1f5eac1115a4 #10785 TUI: fix data-race during resize

CHANGES:

9697c7f56a26 #8194 fix menu_get()
7f2e43c637e5 #9520 improve Lua error messages
c2343180d74f #9526 Remove jemalloc
baf93d96063c #9581 UI: always use concrete colors for default_colors_set
91688b488310 #9563 defaults: set 'scrollback' to -1 by default
bb24fec33355 #10136 defaults: exclude "S" from 'shortmess'
ddd0eb6f5120 #8540 startup: -es/-Es (silent/batch mode): skip swapfile
35362495c965 #9805 jumplist: avoid extra tail entry
939d9053bdf2 #10573 channels: reflect exit due to signals in exit status code
45c34bd84aa5 #10689 :doautocmd : Never show "No matching autocommands"
fb19aeeb33f7 #9110 API: make nvim_win_set_option() set window-global, not buffer-local
abfc8b3257f8 #10778 emsg_multiline: log Vim errors
06d9cc734bf0 #10657 exists("$FOO"): return false for empty env var
6616d1d3e5c9 #10743 win/env: Vim-compat: Empty string deletes env var
7d664837e1a9 #10662 win: expand nested env var in $HOME
2816bc8620ca #8349 edit.c: Disable indent during completion
58f505dc7432 #9829 startup: remove TUI init special-case
    Historically Vim/Nvim does backflips to handle input and show messages
    before a UI is available. This logical contradiction was already fixed
    for remote UIs (#9024 c236e80cf3df). Fixing it also for the TUI avoids
    problems on Windows, simplifies the logic, and avoids races like #9959.

* tests: move os_kill to functional helpers

* tests: fix system_spec when run with clipboard manager

Replaces "xclip" with a dedicated helper program.

Fixes: https://github.com/neovim/neovim/issues/4900#issuecomment-501866842

REP_NODELAY was added because REP delayed too much.  This changes REP to
only add a delay on every 100th line instead.

This helps to cover the additional pulse steps with
out_data_decide_throttle, which would have required to change
REP_NODELAY anyway.

This was overlooked in 8072f085d2ed.

Analogous to 8a782f1699e2.

fix #10668
ref 8072f085d2ed #9884
ref 8a782f1699e2 #2377

Fix flaky "shell command :! throttles shell-command output greater than ~10KB:":

    [ RUN      ] shell command :! throttles shell-command output greater than ~10KB:
    warning: Screen changes were received after the expected state. This indicates
    indeterminism in the test. Try adding screen:expect(...) (or wait()) between
    asynchronous (feed(), nvim_input()) and synchronous API calls.
      - Use screen:redraw_debug() to investigate; it may find relevant intermediate
        states that should be added to the test to make it more robust.
      - If the purpose of the test is to assert state after some user input sent
        with feed(), adding screen:expect() before the feed() will help to ensure
        the input is sent when Nvim is in a predictable state. This is preferable
        to wait(), for being closer to real user interaction.
      - wait() can trigger redraws and consequently generate more indeterminism.
        Try removing wait().

    ERR
    test/functional/ui/screen.lua:579: Failed to match any screen lines.
    Expected (anywhere): "
    %."
    Actual:
      |XXXXXXXXXX 591                                    |
      |XXXXXXXXXX 592                                    |
      |XXXXXXXXXX 593                                    |
      |XXXXXXXXXX 594                                    |
      |                                                  |
      |                                                  |
      |{3:-- TERMINAL --}                                    |
    stack traceback:
            test/functional/ui/screen.lua:579: in function '_wait'
            test/functional/ui/screen.lua:367: in function 'expect'
            test/functional/ui/output_spec.lua:63: in function <test/functional/ui/output_spec.lua:53>

Log: https://travis-ci.org/neovim/neovim/jobs/569082705#L5355
(gcc-functionaltest-lua)

We don't want to retry autoload sourcing (slow) for every random has()
query that finds it way to eval_call_provider().

Value of 1 cannot be used, because users might set that in their vimrc
to _disable_ a provider, which would confuse :checkhealth and has().