fix(vim.ui)!: change open() to return `result|nil, errmsg|nil` #28612

reverts e0d92b9cc20b58179599f53dfa74ca821935a539 #28502

Problem:
`vim.ui.open()` has a `pcall()` like signature, under the assumption
that this is the Lua idiom for returning result-or-error. However, the
`result|nil, errmsg|nil` pattern:
- has precedent in:
  - `io.open`
  - `vim.uv` (`:help luv-error-handling`)
- has these advantages:
  - Can be used with `assert()`:
    ```
    local result, err = assert(foobar())
    ```
  - Allows LuaLS to infer the type of `result`:
    ```
    local result, err = foobar()
    if err then
      ...
    elseif result then
      ...
    end
    ```

Solution:
- Revert to the `result|nil, errmsg|nil` pattern.
- Document the pattern in our guidelines.

1 files changed, 3 insertions, 3 deletions

diff --git a/runtime/lua/vim/lsp/handlers.lua b/runtime/lua/vim/lsp/handlers.lua
index ab4fa52c40..4672d94105 100644
--- a/runtime/lua/vim/lsp/handlers.lua
+++ b/runtime/lua/vim/lsp/handlers.lua
@@ -654,15 +654,15 @@ M[ms.window_showDocument] = function(_, result, ctx, _)
 
   if result.external then
     -- TODO(lvimuser): ask the user for confirmation
-    local ok, cmd_or_err = vim.ui.open(uri)
-    local ret = ok and (cmd_or_err --[[@as vim.SystemObj]]):wait(2000) or nil
+    local cmd, err = vim.ui.open(uri)
+    local ret = cmd and cmd:wait(2000) or nil
 
     if ret == nil or ret.code ~= 0 then
       return {
         success = false,
         error = {
           code = protocol.ErrorCodes.UnknownErrorCode,
-          message = ret and ret.stderr or cmd_or_err,
+          message = ret and ret.stderr or err,
         },
       }
     end