Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Heavy flow of errors with NixD #2390

Open
1 task done
Sk7Str1pe opened this issue Oct 8, 2024 · 19 comments
Open
1 task done

[BUG] Heavy flow of errors with NixD #2390

Sk7Str1pe opened this issue Oct 8, 2024 · 19 comments
Labels
bug Something isn't working

Comments

@Sk7Str1pe
Copy link

Sk7Str1pe commented Oct 8, 2024

Field Description
Plugin lspconfig
Nixpkgs unstable
Home Manager unstable
  • I have read the FAQ and my bug is not listed there.

Description

image
error i get every time editing configuration.nix

Logs

[START][2024-10-08 20:06:15] LSP logging initiated
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.636] 16509: <-- initialize(1)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.636] 16509: --> reply:initialize(1)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.637] 16509: --> call window/workDoneProgress/create(1)\nI[20:06:15.637] 16509: --> notify $/progress\nI[20:06:15.637] 16509: --> call attrset/evalExpr(1)\nI[20:06:15.637] 16509: --> call window/workDoneProgress/create(2)\nI[20:06:15.637] 16509: --> notify $/progress\nI[20:06:15.637] 16509: --> call attrset/evalExpr(1)\nI[20:06:15.637] 16509: --> call workspace/configuration(3)\nI[20:06:15.638] 16509: <-- initialized\nI[20:06:15.638] 16509: <-- textDocument/didOpen\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.639] 16509: --> notify textDocument/publishDiagnostics\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.639] 16509: <-- textDocument/inlayHint(2)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.639] 16509: <-- textDocument/inlayHint(3)\nI[20:06:15.639] 16509: <-- reply(1)\nI[20:06:15.639] 16509: --> call attrset/attrpathInfo(2)\nI[20:06:15.639] 16509: --> call attrset/attrpathInfo(3)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.639] 16509: <-- reply(2)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.640] 16509: <-- reply(3)\nE[20:06:15.640] 16509: workspace/configuration: parse error expected object\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.640] 16509: <-- textDocument/semanticTokens/full(4)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.643] 16509: --> reply:textDocument/semanticTokens/full(4)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.793] "
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"16509: <-- reply(1)\nI[20:06:15.793] 16509: --> notify $/progress\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.794] 16509: <-- reply(2)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.794] 16509: <-- reply(3)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I["
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"20:06:15.794] 16509: --> reply:textDocument/inlayHint(2)\n"
[ERROR][2024-10-08 20:06:15] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:15.794] 16509: --> reply:textDocument/inlayHint(3)\n"
[ERROR][2024-10-08 20:06:16] .../vim/lsp/rpc.lua:770	"rpc"	"/nix/store/2dx04iayp5nl6zwsa60dsm82c58ghwk7-nixd-2.4.0/bin/nixd"	"stderr"	"I[20:06:16.157] 16509: <-- reply(1)\nI[20:06:16.157] 16509: --> notify $/progress\n"

Minimal, Reproducible Example (MRE)

programs.nixvim = {
  enable = true;
  lsp = {
	    enable = true;
	    inlayHints = true;
	    servers = {
	      lua-ls = {
	        enable = true;
	      };
	      pylsp = {
	        enable = true;
	      };
	      nixd = {
	        enable = true;
	      };
	      clangd = {
	        enable = true;
	      };
	    };
	  };
}

also nixd installed with environment.systemPackages. except flow of errors, lsp works fine

@Sk7Str1pe Sk7Str1pe added the bug Something isn't working label Oct 8, 2024
@GaetanLepage
Copy link
Member

@inclyc do you think this issue is on our side ?
FWIW, neovim has been updated to 0.10.2, but I would be surprised if it would break something like this.

@inclyc
Copy link
Member

inclyc commented Oct 9, 2024

According to nixd's log I think it works as expected, can you take a look? @GaetanLepage

@Sk7Str1pe
Copy link
Author

Yes, as I said lsp works fine

@khaneliman
Copy link
Contributor

Sounds similar to report in #2290 . It doesn't seem to affect everyone so I'm not sure of the root cause, yet.

@Sk7Str1pe
Copy link
Author

UPD someone figured out that issues appears only if nerdfont's glyths are used in file

@MattSturgeon
Copy link
Member

issues appears only if nerdfont's glyths are used in file

Interesting, I haven't had the issue with other unicode characters... I do sometimes include emoji in my nix comments.

It is worth noting that nix strings themselves don't have UTF-8 support, so if nix evaluates a unicode string you can run into issues. IDK if nixd actually evaluates any strings using nix, but maybe they have similar bugs on their end?

@Lassulus
Copy link

for me this happens (recently? not sure I had this before) when editing files which have unicode in them. In a nix config I have this line:
https://github.com/Lassulus/superconfig/blob/master/tools/nvim.nix#L26

listchars = "tab:⇥ ,extends:❯,precedes:❮,nbsp:␣,trail:·,eol:↲";

and now nixvim with nixd throws errors when opening the file which make editing pretty hard.

@eljamm
Copy link

eljamm commented Oct 11, 2024

I can reproduce this issue with neovim 0.10.2, but everything works fine with 0.10.0 and 0.10.1.

@Sk7Str1pe
Copy link
Author

So this is neovim problem? Gotta open issue in their repo.

@eljamm
Copy link

eljamm commented Oct 11, 2024

Yes, I've bisected the issue to neovim/neovim@b55435f

bisect log

git bisect start
# status: waiting for both good and bad commits
# good: [7e194f0d0c33a0a1b7ccfaf2baafdacf7f22fbb5] NVIM 0.10.1
git bisect good 7e194f0d0c33a0a1b7ccfaf2baafdacf7f22fbb5
# status: waiting for bad commit, 1 good commit known
# bad: [8b98642002d0506d20628683958cb5c97a0dad80] NVIM 0.10.2
git bisect bad 8b98642002d0506d20628683958cb5c97a0dad80
# good: [1fe8dc4d2c4e464684bfa05350e69c47c31431e7] Merge pull request #30211 from neovim/backport-30209-to-release-0.10
git bisect good 1fe8dc4d2c4e464684bfa05350e69c47c31431e7
# bad: [ba47b440fdf975a26c267337bc3a56c75349488e] fix(treesitter): lint top-level anonymous nodes
git bisect bad ba47b440fdf975a26c267337bc3a56c75349488e
# bad: [6f34f07a3169b05af57d06428fca23c33354f2d2] ci: enable automerge by default when backporting
git bisect bad 6f34f07a3169b05af57d06428fca23c33354f2d2
# good: [461ed5e6bd864ea5d5c37b86bc43728071af5d06] Merge pull request #30303 from neovim/backport-30300-to-release-0.10
git bisect good 461ed5e6bd864ea5d5c37b86bc43728071af5d06
# bad: [ef05c514aafb1eda4081f5f0408e93da0f7639dc] fix(runtime): add remaining missing commentstrings (#30252)
git bisect bad ef05c514aafb1eda4081f5f0408e93da0f7639dc
# bad: [e13564b9f44ec7b188c3010d2b3c4a3a3e405e6e] fix(lsp): handle encoding bounds in str_utfindex_enc
git bisect bad e13564b9f44ec7b188c3010d2b3c4a3a3e405e6e
# bad: [b55435f4386e9a138be7275919aa784f79a2fb03] fix(lsp): handle out-of-bounds character positions #30288
git bisect bad b55435f4386e9a138be7275919aa784f79a2fb03
# first bad commit: [b55435f4386e9a138be7275919aa784f79a2fb03] fix(lsp): handle out-of-bounds character positions #30288

@eljamm
Copy link

eljamm commented Oct 11, 2024

So this is neovim problem? Gotta open issue in their repo.

There is one already open in neovim/neovim#30675

The workaround in the issue above seems to be working as well. Just add offset_encoding = 'utf-8', to nixd's lspconfig:

NeoVim

local nvim_lsp = require 'lspconfig'
nvim_lsp.nixd.setup {
  offset_encoding = 'utf-8',
  settings = {
    ...
  },
}

NixVim

plugins.lsp.servers.nixd = {
  enable = true;
  extraOptions = {
    offset_encoding = "utf-8";
  };
};

@GaetanLepage
Copy link
Member

Thanks for the investigation @eljamm !
Should we set offset_encoding = lib.mkDefault "utf-8"; or is this too opinionated (or simply not our concern) ?

@Sk7Str1pe
Copy link
Author

So I set offset_encoding = "utf-8" And I have no errors but syntax highlights changed a lot

@Sk7Str1pe
Copy link
Author

image
before VS after

@eljamm
Copy link

eljamm commented Oct 14, 2024

So I set offset_encoding = "utf-8" And I have no errors but syntax highlights changed a lot

I didn't notice at first as both colors are similar in my colorscheme. Interestingly, this also doesn't happen right after opening the file. To trigger it, I need to insert a newline under a line which has a glyph in it. Editing all the other lines is fine.

Should we set offset_encoding = lib.mkDefault "utf-8"; or is this too opinionated (or simply not our concern) ?

If it weren't for the broken syntax highlighting, I'd say we go for it, but perhaps it would be better to wait until we can patch this. I don't know how many people are affected, but they'll probably find this issue and the workaround and choose if it's worth to use it or just roll back Neovim.

@MattSturgeon
Copy link
Member

Thanks for the investigation @eljamm ! Should we set offset_encoding = lib.mkDefault "utf-8"; or is this too opinionated (or simply not our concern) ?

Looking at the lua above, it seems this would need to be set in plugins.lsp.settings instead of plugins.lsp.servers.nixd.settings? While it isn't our default policy, I think I would personally like to set it as a default. At most, perhaps have a default-on option that does it.

So I set offset_encoding = "utf-8" and [...] syntax highlights changed a lot

before VS after

I can see differences between left & right, but I can't see any issues in your screenshot. At first glance it just looks like the right hand side is using LSP-based syntax highlighting while the left hand side is not.

Your left hand side also doesn't seem to be using treesitter highlighting,1 which may explain why the difference is so stark?

Footnotes

  1. plugins.treesitter.settings.highlight.enable

@stasjok
Copy link
Contributor

stasjok commented Oct 17, 2024

Looking at the lua above, it seems this would need to be set in plugins.lsp.settings instead of plugins.lsp.servers.nixd.settings? While it isn't our default policy, I think I would personally like to set it as a default. At most, perhaps have a default-on option that does it.

I don't recommend setting it globally, it can cause troubles in other servers. AFAIK content encoding is negotiated between the server and the client, but only since lsp spec 3.17 Before it was always utf-16.. And spec states that support for utf-16 is mandatory. By forcing it to utf-8 on neovim side encoding can be mismatched between the client and the server in some cases. This issue was a regression that was backported in 0.10. It should be fixed now in 0.10 branch, but I don't know when 0.10.3 will be released. Maybe a temporary patch can be included to nixpkgs: neovim/neovim@84bbbd9. Since it's the bug in Nvim, not in Nixvim, I don't think here is the place for workarounds line that, that are valuable only for one specific version of Neovim (0.10.2).

eljamm added a commit to eljamm/nixpkgs that referenced this issue Oct 17, 2024
Byte index encoding bounds were previously only checked against the
UTF-8 length, whereas the default is UTF-16, which caused undefined
behaviour.

neovim/neovim#30747
nix-community/nixvim#2390
eljamm added a commit to eljamm/nixpkgs that referenced this issue Oct 17, 2024
Byte index encoding bounds were previously only checked against the
UTF-8 length, whereas the default is UTF-16, which caused undefined
behaviour.

neovim/neovim#30747
nix-community/nixvim#2390
eljamm added a commit to eljamm/nixpkgs that referenced this issue Oct 17, 2024
Fix byte index encoding bounds, which were previously only checked
against the UTF-8 length, whereas the default is UTF-16, which caused
undefined behaviour.

See:
- neovim/neovim#30747
- nix-community/nixvim#2390
@khaneliman
Copy link
Contributor

Have you tried using nightly neovim to see if it resolves your issues? I was able to use nixd with it (I had other plugins I needed to work so I rolled back to stable)

@MattSturgeon
Copy link
Member

MattSturgeon commented Oct 19, 2024

For anyone who missed it, there's also an overlay suggested here that you can use to apply the fix to neovim 0.10:

  (final: prev: {
    neovim-unwrapped = prev.neovim-unwrapped.overrideAttrs (old: {
      patches = old.patches ++ [
        # Fix byte index encoding bounds.
        # - https://github.com/neovim/neovim/pull/30747
        # - https://github.com/nix-community/nixvim/issues/2390
        (final.fetchpatch {
          name = "fix-lsp-str_byteindex_enc-bounds-checking-30747.patch";
          url = "https://patch-diff.githubusercontent.com/raw/neovim/neovim/pull/30747.patch";
          hash = "sha256-2oNHUQozXKrHvKxt7R07T9YRIIx8W3gt8cVHLm2gYhg=";
        })
      ];
    });
  })

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

8 participants