Full mouse control #1493
Labels
Comments
|
We're only a client so there is limits to what extra features we can add. Do you know of any VNC client/server combination where this already works? |
|
RealVNC seems to support this, or something like this, as Minecraft Pi Edition seems to work over their VNC (see here). I know that RealVNC is proprietary, though, and I think it requires their client as well. |
|
Unfortunately the fact that they are proprietary does mean it doesn't help us that they have that functionality. We need to know how the extension works, either through documentation or source code. |
|
x11vnc has a option called |
|
This could in theory be achieved through the Pointer Lock API (demo). Are patches welcome for this? If so, I can maybe cook something up. |
lhchavez
added a commit
to lhchavez/noVNC
that referenced
this issue
Feb 8, 2021
This change adds the following: a) A new button on the UI to enter full pointer lock mode, which invokes the Pointer Lock API[1] on the canvas, which hides the cursor and makes mouse events provide relative motion from the previous event (through `movementX` and `movementY`). These can be added to the previously-known mouse position to convert it back to an absolute position. b) Adds support for the VMware Cursor Position pseudo-encoding[2], which servers can use when they make cursor position changes themselves. This is done by some APIs like SDL, when they detect that the client does not support relative mouse movement[3] and then "warp"[4] the cursor to the center of the window, to calculate the relative mouse motion themselves. c) When the canvas is in pointer lock mode and the cursor is not being locally displayed, it updates the cursor position with the information that the server sends, since the actual position of the cursor does not matter locally anymore, since it's not visible. d) Adds some tests for the above. You can try this out end-to-end with TigerVNC with TigerVNC/tigervnc#1198 applied! Fixes: novnc#1493 under some circumstances (at least all SDL games would now work). 1: https://developer.mozilla.org/en-US/docs/Web/API/Pointer_Lock_API 2: https://github.com/rfbproto/rfbproto/blob/master/rfbproto.rst#vmware-cursor-position-pseudo-encoding 3: https://hg.libsdl.org/SDL/file/28e3b60e2131/src/events/SDL_mouse.c#l804 4: https://tronche.com/gui/x/xlib/input/XWarpPointer.html
lhchavez
added a commit
to lhchavez/noVNC
that referenced
this issue
Feb 8, 2021
This change adds the following: a) A new button on the UI to enter full pointer lock mode, which invokes the Pointer Lock API[1] on the canvas, which hides the cursor and makes mouse events provide relative motion from the previous event (through `movementX` and `movementY`). These can be added to the previously-known mouse position to convert it back to an absolute position. b) Adds support for the VMware Cursor Position pseudo-encoding[2], which servers can use when they make cursor position changes themselves. This is done by some APIs like SDL, when they detect that the client does not support relative mouse movement[3] and then "warp"[4] the cursor to the center of the window, to calculate the relative mouse motion themselves. c) When the canvas is in pointer lock mode and the cursor is not being locally displayed, it updates the cursor position with the information that the server sends, since the actual position of the cursor does not matter locally anymore, since it's not visible. d) Adds some tests for the above. You can try this out end-to-end with TigerVNC with TigerVNC/tigervnc#1198 applied! Fixes: novnc#1493 under some circumstances (at least all SDL games would now work). 1: https://developer.mozilla.org/en-US/docs/Web/API/Pointer_Lock_API 2: https://github.com/rfbproto/rfbproto/blob/master/rfbproto.rst#vmware-cursor-position-pseudo-encoding 3: https://hg.libsdl.org/SDL/file/28e3b60e2131/src/events/SDL_mouse.c#l804 4: https://tronche.com/gui/x/xlib/input/XWarpPointer.html
lhchavez
added a commit
to lhchavez/noVNC
that referenced
this issue
Feb 8, 2021
This change adds the following: a) A new button on the UI to enter full pointer lock mode, which invokes the Pointer Lock API[1] on the canvas, which hides the cursor and makes mouse events provide relative motion from the previous event (through `movementX` and `movementY`). These can be added to the previously-known mouse position to convert it back to an absolute position. b) Adds support for the VMware Cursor Position pseudo-encoding[2], which servers can use when they make cursor position changes themselves. This is done by some APIs like SDL, when they detect that the client does not support relative mouse movement[3] and then "warp"[4] the cursor to the center of the window, to calculate the relative mouse motion themselves. c) When the canvas is in pointer lock mode and the cursor is not being locally displayed, it updates the cursor position with the information that the server sends, since the actual position of the cursor does not matter locally anymore, since it's not visible. d) Adds some tests for the above. You can try this out end-to-end with TigerVNC with TigerVNC/tigervnc#1198 applied! Fixes: novnc#1493 under some circumstances (at least all SDL games would now work). 1: https://developer.mozilla.org/en-US/docs/Web/API/Pointer_Lock_API 2: https://github.com/rfbproto/rfbproto/blob/master/rfbproto.rst#vmware-cursor-position-pseudo-encoding 3: https://hg.libsdl.org/SDL/file/28e3b60e2131/src/events/SDL_mouse.c#l804 4: https://tronche.com/gui/x/xlib/input/XWarpPointer.html
|
Would really love this functionality! Any updates @lhchavez @samhed @CendioOssman @zeroxoneafour @jswanner ? |
|
I'm also looking for a fix for this. I use noVNC as a remote desktop for while im out and it causes some issues with programs that require relative mouse movement. |
aseering
pushed a commit
to aseering/noVNC
that referenced
this issue
Feb 20, 2023
This change adds the following: a) A new button on the UI to enter full pointer lock mode, which invokes the Pointer Lock API[1] on the canvas, which hides the cursor and makes mouse events provide relative motion from the previous event (through `movementX` and `movementY`). These can be added to the previously-known mouse position to convert it back to an absolute position. b) Adds support for the VMware Cursor Position pseudo-encoding[2], which servers can use when they make cursor position changes themselves. This is done by some APIs like SDL, when they detect that the client does not support relative mouse movement[3] and then "warp"[4] the cursor to the center of the window, to calculate the relative mouse motion themselves. c) When the canvas is in pointer lock mode and the cursor is not being locally displayed, it updates the cursor position with the information that the server sends, since the actual position of the cursor does not matter locally anymore, since it's not visible. d) Adds some tests for the above. You can try this out end-to-end with TigerVNC with TigerVNC/tigervnc#1198 applied! Fixes: novnc#1493 under some circumstances (at least all SDL games would now work). 1: https://developer.mozilla.org/en-US/docs/Web/API/Pointer_Lock_API 2: https://github.com/rfbproto/rfbproto/blob/master/rfbproto.rst#vmware-cursor-position-pseudo-encoding 3: https://hg.libsdl.org/SDL/file/28e3b60e2131/src/events/SDL_mouse.c#l804 4: https://tronche.com/gui/x/xlib/input/XWarpPointer.html
|
I find that turning off Raw Input in Mouse Settings makes it a bit easier to play MineCraft (and of course, use a mouse, not a track pad). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
When I use noVNC for remote gaming, games that require mouse control (such as first-person games like Minecraft) cannot interpret the mouse input correctly, most likely because the application in question is not given full mouse control.
Describe the solution you'd like
It would be great if noVNC was able to fully immerse the mouse in the VNC using mouse control. First-person games online are able to "take control of your pointer" and it would be nice if there was a setting to allow that in the options.
Describe alternatives you've considered
I tried reconfiguring Minecraft's settings, but nothing works.
Additional context
When mouse control is fully taken, it would also probably be a good idea to make a keybind (such as esc) to give mouse control back to the client computer.
The text was updated successfully, but these errors were encountered: