Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add OTG mode (HID only) #2974

Merged
merged 20 commits into from
Jan 27, 2022
Merged

Add OTG mode (HID only) #2974

merged 20 commits into from
Jan 27, 2022

Conversation

rom1v
Copy link
Collaborator

@rom1v rom1v commented Jan 26, 2022 •
edited
Loading

Add an option --otg to run scrcpy with only physical keyboard (#2632) and mouse (#2916) simulation, without mirroring and without requiring adb (USB debugging), as if the computer keyboard and mouse were plugged directly to the device via an OTG cable.

scrcpy --otg
# Pass the serial if several USB devices are available
scrcpy --otg -s 0123456789abcdef

This opens a window (with just the scrcpy logo, there is no mirroring), which when focused forwards the keyboard and mouse to the device:

otg

The mouse capture works the same way as in "normal" mode (as explained in #2916):

  • when the mouse is "captured", the mouse pointer disappears from the computer and appears on the Android device
  • a click in the window captures the mouse
  • special captures keys, either Alt or Super, toggle (disable or enable) the mouse capture (use one of them to give the control of the mouse back to the computer)

Like --hid-keyboard and --hid-mouse, --otg only works if the device is connected by USB, and is currently only supported on Linux.

@lucas-bortoli
Copy link

This is awesome! Nice work!

@rom1v rom1v force-pushed the dev branch 2 times, most recently from 7e8a942 to c0de365 Compare January 27, 2022 18:02
rom1v added 20 commits January 27, 2022 23:35
@rom1v
Move AOA/HID code to usb/
2762f5d 
PR #2974 <#2974>
@rom1v
Rename HAVE_AOA_HID to HAVE_USB
d48d191 
The condition actually determines whether scrcpy can use libusb or not.

PR #2974 <#2974>
@rom1v
Extract USB handling from AOA
1d6f995 
The AOA code handled both USB initialization and AOA commands/events.
Extract USB-related code to a separate file and structure.

PR #2974 <#2974>
@rom1v
Make serial mandatory for sc_usb
48e3ff2 
In practice, it is already mandatory.

PR #2974 <#2974>
@rom1v
Move sc_usb out of sc_aoa
adda47b 
This will allow to initialize a USB device separately and pass it to
sc_aoa.

PR #2974 <#2974>
@rom1v
Remove libusb_device field
b779eca 
It is possible to retrieve the device instance from the handle via
libusb_get_device(), so we don't need to reference the device one more
time.

PR #2974 <#2974>
@rom1v
Find device with USB context
2114f48 
An explicit context was used everywhere except for listing the devices.

PR #2974 <#2974>
@rom1v
Split USB initialization and connection
bbef426 
This will allow to execute other USB calls (retrieving the device list
for example) before connecting to the selected device.

PR #2974 <#2974>
@rom1v
Add util function to read USB descriptor string
1ab3692 
Use it from accept_device() to simplify (at the cost an additional
allocation for each serial, but it is not important).

It will also be useful in other functions in further commits.

PR #2974 <#2974>
@rom1v
Expose function to find a USB device
0ee9e2f 
The device was automatically found by sc_usb_connect(). Instead, expose
a function to find a device from a serial, and let the caller connect to
the device found (if any).

This will allow to list all devices first, then select one device to
connect to.

PR #2974 <#2974>
@rom1v
Wrap libusb_device
d8b37fe 
Introduce a structure to wrap a libusb_device and expose its descriptor
data read during discovery.

PR #2974 <#2974>
@rom1v
Find a list of devices instead of a single one
1c17f57 
Several devices may match the requested serial, but above all, this
paves the way to list all devices (when no serial is provided).

PR #2974 <#2974>
@rom1v
Make serial optional to find USB devices
8fc9dca 
If no serial is provided, then list all available USB devices (which can
be open and having a serial).

PR #2974 <#2974>
@rom1v
Make acksync optional for AOA initialization
37987b8 
Acksync is used to delay HID events until some request (in practice,
device clipboard synchronization) is acknowledged by the device.

This mechanism will not be necessary for OTG mode.

PR #2974 <#2974>
@rom1v
Detect USB device disconnection
1a03206 
The device disconnection is detected when the video socket closes.

In order to introduce an OTG mode (HID events) without mirroring (and
without server), we must be able to detect USB device disconnection.

This feature will only be used in OTG mode.

PR #2974 <#2974>
@rom1v
Move input event helpers
36aaf70 
Input events helpers to convert from SDL events to scrcpy events were
implemented in input_manager. To reuse them for OTG mode, move them to
input_events.h.

PR #2974 <#2974>
@rom1v
Add OTG mode
91418c7 
Add an option --otg to run scrcpy with only physical keyboard and mouse
simulation (HID over AOA), without mirroring and without requiring adb.

To avoid adding complexity into the scrcpy initialization and screen
implementation, OTG mode is implemented totally separately, with a
separate window.

PR #2974 <#2974>
@rom1v
Document OTG mode in README
c5be0d6 
PR #2974 <#2974>
@rom1v
Make HID keyboard and mouse optional in OTG mode
ea68a00 
Allow to only enable HID keyboard or HID mouse:

    scrcpy --otg -K   # keyboard only
    scrcpy --otg -M   # mouse only
    scrcpy --otg -KM  # keyboard and mouse
    scrcpy --otg      # keyboard and mouse

PR #2974 <#2974>
@rom1v
Enable mouse focus clickthrough in OTG mode
5508c63 
A single click on the window must both give focus and capture the mouse.

PR #2974 <#2974>
@rom1v rom1v merged commit 5508c63 into dev Jan 27, 2022
@BachoSeven
Copy link

@rom1v Would it be possible to also add mirroring to this OTG mode which doesn't require USB debugging? This would be especially useful for broken screens :'D

thanks in advance

@rom1v
Copy link
Collaborator Author

rom1v commented Feb 5, 2022

@BachoSeven This is not possible without installing an app on the device and authorizing capture, so that would not work with a broken screen.

@vielavlrs
Copy link

vielavlrs commented Feb 12, 2022 •
edited
Loading

I tried installing scrcpy v1.22 on Ubuntu 20.04 thru snap but the terminal shows my scrcpy is still v1.12? How do I run v1.22 in Ubuntu 20.04?

@rom1v
Copy link
Collaborator Author

rom1v commented Feb 12, 2022

I tried installing scrcpy v1.22 on Ubuntu 20.04 but the terminal shows my scrcpy is still v1.12? How do I run v1.22 in Ubuntu 20.04?

Uninstall the one from the distribution (apt remove scrcpy), and follow these steps to install the latest release.

@Yuki-J1 Yuki-J1 mentioned this pull request Feb 22, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@rom1v @lucas-bortoli @BachoSeven @vielavlrs