fix(vim.ui)!: change open() to return `result|nil, errmsg|nil` #28612

reverts e0d92b9cc20b58179599f53dfa74ca821935a539 #28502

Problem:
`vim.ui.open()` has a `pcall()` like signature, under the assumption
that this is the Lua idiom for returning result-or-error. However, the
`result|nil, errmsg|nil` pattern:
- has precedent in:
  - `io.open`
  - `vim.uv` (`:help luv-error-handling`)
- has these advantages:
  - Can be used with `assert()`:
    ```
    local result, err = assert(foobar())
    ```
  - Allows LuaLS to infer the type of `result`:
    ```
    local result, err = foobar()
    if err then
      ...
    elseif result then
      ...
    end
    ```

Solution:
- Revert to the `result|nil, errmsg|nil` pattern.
- Document the pattern in our guidelines.

1 files changed, 1 insertions, 1 deletions

diff --git a/test/functional/lua/ui_spec.lua b/test/functional/lua/ui_spec.lua
index 3f5c051347..d69e893c96 100644
--- a/test/functional/lua/ui_spec.lua
+++ b/test/functional/lua/ui_spec.lua
@@ -144,7 +144,7 @@ describe('vim.ui', function()
       end
       if not is_os('bsd') then
         local rv =
-          exec_lua [[local _, cmd = vim.ui.open('non-existent-file'); return cmd:wait(100).code]]
+          exec_lua [[local cmd = vim.ui.open('non-existent-file'); return cmd:wait(100).code]]
         ok(type(rv) == 'number' and rv ~= 0, 'nonzero exit code', rv)
       end