Add HID mouse support #2916
Add HID mouse support #2916
Conversation
I suggest to use USB OTG cable to connect your mouse to Android directly and see if it works in firefox, maybe it's firefox or Android's bug. |
|
@AlynxZhou Thank you for your suggestion, I could find my OTG cable right now, but it makes me remember that I could just pair my bluetooth mouse, so I can test. And indeed, button 5 does nothing in Firefox. |
This avoids to directly pass the options instance (which contains more data than strictly necessary), and limit the number of parameters for the init function.
The input_manager is strongly tied to the screen, it could not work
independently of the specific screen implementation.
To implement a user-friendly HID mouse behavior, some SDL events
will need to be handled both by the screen and by the input manager. For
example, a click must typically be handled by the input_manager so that
it is forwarded to the device, but in HID mouse mode, the first click
should be handled by the screen to capture the mouse (enable relative
mouse mode).
Make the input_manager a descendant of the screen, so that the screen
decides what to do on SDL events.
Concretely, replace this structure hierarchy:
+- struct scrcpy
+- struct input_manager
+- struct screen
by this one:
+- struct scrcpy
+- struct screen
+- struct input_manager
This allows to merge two "#ifdef HAVE_AOA_HID" blocks to simplify.
The AOA initialization state is already tracked by aoa_hid_initialized.
If AOA or HID keyboard may not be initialized for some reason, acksync is useless.
A scroll event might be produced when a mouse button is pressed (for example when scrolling while selecting a text). For consistency, pass the actual buttons state (instead of 0). In practice, it seems that this use case does not work properly with Android event injection, but it will work with HID mouse.
The default mouse injection works in absolute mode: it forwards clicks at a specific position on screen. To support HID mouse, add a flag to indicate that the mouse processor works in relative mode: it forwards mouse motion vectors, without any absolute reference to the screen.
This will allow the mouse processor to handle relative motion easily.
The decision to not send motion events when no click is pressed is specific to Android mouse injection. Other mouse processors (e.g. for HID mouse) will need to receive all events.
In relative mouse mode, the mouse pointer must be "captured" from the computer. Toggle (disable/enable) relative mouse mode using any of the hardcoded capture keys: - left-Alt - left-Super - right-Super These capture keys do not conflict with shortcuts, since a shortcut is always a combination of the MOD key and some other key, while the capture key triggers an action only if it is pressed and released alone. The relative mouse mode is also automatically enabled on any click in the window, and automatically disabled on focus lost (it is possible to lose focus even without the mouse).
This paves the way to add support for HID mouse initialization.
|
А можно ли убрать кнопку назад ( правый мышь нажатия ) |
|
Please speak english. Yes, you can forward all clicks with |
Thanks for the information |
|
Let's merge it 🐭 |
|
By any mean... will this come to windows? at least with the Linux subsystem? :) |
|
" the mouse pointer disappears from the computer and appears on the Android device." Not sure I understand. The mouse cursor? It gets to appear on the device? How does it look like? And what about pressing it? |
It works as if you plugged your mouse on the device directly. So you get a cursor on the physical device, you could click, etc. |
|
@rom1v Cool. Would it show some indication when you click, though? |
|
Hope it comes to Windows. If I recall correctly this feature (both mouse & keyboard) is available in Samsung's Sidesync (I don't know how it works though) and it works on Windows, but I guess they discontinued the software and no more update is available. |
it's known as Samsung Flow now and also it's gonna go away too I guess... since now there are both DEX and also Your Phone app on windows... but this implementation from scrcpy is way much better than those... I hope this will be available for windows :) |
Currently it does not work because libusb does not work well on Windows, and for Linux devs, Windows is a horrible & unfriendly platform, so I cannot figure out how to make it work. |
yeah.... that's unfortunate but home one day... it will hppn :) |
Similar to the
--hid-keyboard(or-K) added in scrcpy v1.20 (#2632), add a new option--hid-mouse(or-M).Basically, you can run
scrcpywith both HID keyboard and mouse with:The mouse is "captured": the mouse pointer disappears from the computer and appears on the Android device.
Special captures keys, either Alt or Super, toggle (disable or enable) the mouse capture. Use one of them to give the control of the mouse back to the computer.
These capture keys do not conflict with shortcuts, since a shortcut is always a combination of the MOD key and some other key, while the capture key triggers an action only if it is pressed and released alone.
Like HID keyboard, it works only when the device is connected via USB, and is only supported on Linux.
Tests and feedback welcome.
cc @AlynxZhou (author of #2632)
cc @JonasLoeffelholz (author of #2653)
cc @chldbwnstm (#2632 (comment))
This PR is based on previous refactors I already pushed to
devbranch to support the feature properly.When run with
-KM --forward-all-clicks, on Firefox/Android, mouse button 4 works as expected (previous page), but mouse button 5 does not (I'd expect to go to the next page). I don't know if I did something wrong or if it is not expected to go to the next page.