Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add release static builds for Linux (and macOS?) #5515

Merged
merged 16 commits into from
Nov 24, 2024
Merged

Add release static builds for Linux (and macOS?) #5515

merged 16 commits into from
Nov 24, 2024

Conversation

rom1v
Copy link
Collaborator

@rom1v rom1v commented Nov 22, 2024

This is the continuation of my work on #5306.

scrcpy-github-actions

This PR make it possible to build a static release for Linux (x86_64).

For that purpose:

  • release.mk is replaced by individual release scripts
  • the dependencies script are made more flexible to select native/cross and static/shared
  • the GitHub Actions script is improved (cf screenshot)
  • a Linux build script is added

Please test the binaries generated by GitHub Actions: #5515 (comment)

old Here is a Linux binary (x86_64) built by GitHub Actions:

Now, I want to do the same for macOS, but I have no mac. So I started a skeleton (the last commit of this PR), I need your help to complete it and make it work. You can read the commit which adds the static build for Linux as a base.

Thank you!

@rom1v rom1v changed the base branch from master to dev November 22, 2024 22:28
@Genxster1998
Copy link
Contributor

For macos , macos-latest runner is arch arm64 only.

@muvaf
Copy link
Contributor

muvaf commented Nov 23, 2024

@rom1v For macOS, I had to fix just small two things but I do see some logs about some missing static libraries #5517

@muvaf
Copy link
Contributor

muvaf commented Nov 23, 2024

@rom1v I can also confirm the build you attached works in Linux amd64:

~/Downloads/scrcpy-linux-ga                                                                                                       06:57:27 PM
❯ ./scrcpy_bin
scrcpy 2.7 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     --> (tcpip)  localhost:37105                 device  <redacted>
/home/muvaf/Downloads/scrcpy-linux-ga/scrcpy-server: 1 file pushed, 0 skipped. 200.1 MB/s (90228 bytes in 0.000s)
[server] INFO: Device: <redacted> (Android 14)
INFO: Renderer: opengl
INFO: OpenGL version: 4.6.0 NVIDIA 560.35.03
INFO: Trilinear filtering enabled
INFO: Texture: 1080x1920

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2024

@muvaf Thank you!

Can you please test branch macos_build macos_build.2 (based on your work on #5517)?

scrcpy-github-actions2

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2024

I can also confirm the build you attached works in Linux amd64

❯ ./scrcpy_bin

Could you also please test with ./scrcpy (so that it uses adb and scrcpy-server from the package)?

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2024

@Genxster1998

For macos , macos-latest runner is arch arm64 only.

What other runner should we use to build a version for older macos? macos-14? They don't provide runners for < 14 apparently: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners

@muvaf
Copy link
Contributor

muvaf commented Nov 23, 2024

Could you also please test with ./scrcpy (so that it uses adb and scrcpy-server from the package)?

In Linux amd64, ./scrcpy works only if I change the last line from scrcpy "$@" to ./scrcpy_bin "$@".

❯ ./scrcpy
scrcpy 2.7 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     --> (tcpip)  emulator-5554                   device  sdk_gphone64_x86_64
./scrcpy-server: 1 file pushed, 0 skipped. 213.4 MB/s (90228 bytes in 0.000s)
[server] INFO: Device: [Google] google sdk_gphone64_x86_64 (Android 15)
[server] ERROR: Encoding error: java.lang.IllegalArgumentException: 
[server] INFO: Retrying with -m2560...
INFO: Renderer: opengl
INFO: OpenGL version: 4.6.0 NVIDIA 560.35.03
INFO: Trilinear filtering enabled
INFO: Texture: 1344x2992
INFO: Texture: 1152x2560

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2024

In Linux amd64, ./scrcpy works only if I change the last line from scrcpy "$@" to ./scrcpy_bin "$@".

🤦 Thank you, I fixed locally in app/data/scrcpy_static_wrapper.sh. 👍

@muvaf
Copy link
Contributor

muvaf commented Nov 23, 2024

Can you please test branch macos_build macos_build.2 (based on your work on #5517)?

@rom1v Worked as is!

Similarly, I needed to change the script.

> ./scrcpy
scrcpy 2.7 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     --> (tcpip)  emulator-5554                   device  sdk_gphone64_arm64
./scrcpy-server: 1 file pushed, 0 skipped. 236.4 MB/s (71200 bytes in 0.000s)
[server] INFO: Device: [Google] google sdk_gphone64_arm64 (Android 15)
ERROR: Could not get icon path
WARN: Could not load icon
INFO: Renderer: metal
INFO: Texture: 1080x2400
2024-11-23 19:26:07.856 scrcpy_bin[22331:1443939] +[IMKClient subclass]: chose IMKClient_Modern
2024-11-23 19:26:07.856 scrcpy_bin[22331:1443939] +[IMKInputSession subclass]: chose IMKInputSession_Modern
WARN: Killing the server...


> cat scrcpy
#!/bin/bash
cd "$(dirname ${BASH_SOURCE[0]})"
export ADB=./adb
export SCRCPY_SERVER_PATH=./scrcpy-server
export ICON=./icon.png
./scrcpy_bin "$@"

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2024

For the linux static build, I get this warning:

WARNING: Static library 'udev' not found for dependency 'libusb-1.0', may not be statically linked

I think this is ok (it just requires udev to be installed at runtime).

@muvaf
Copy link
Contributor

muvaf commented Nov 23, 2024

@rom1v What I'm not sure is that when it says udev not found for dependency libusb-1.0, will it require udev to be present or libusb-1.0 ? If the former, then the macOS ones are also OK because they are also system libraries. But if it means libusb-1.0 is completely out, then that breaks the purpose.

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2024

What I'm not sure is that when it says udev not found for dependency libusb-1.0, will it require udev to be present or libusb-1.0

Just udev:

$ ldd scrcpy_bin 
	linux-vdso.so.1 (0x00007f4b39692000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f4b37e7c000)
	libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007f4b37e37000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4b37c41000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f4b39694000)
	libcap.so.2 => /lib/x86_64-linux-gnu/libcap.so.2 (0x00007f4b37c35000)

@muvaf
Copy link
Contributor

muvaf commented Nov 23, 2024

@rom1v This is what I see in macOS with the current macos_build.2 branch:

> otool -L release/work/build-macos/dist/scrcpy_bin
release/work/build-macos/dist/scrcpy_bin:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 3107.0.0)
	/System/Library/Frameworks/CoreVideo.framework/Versions/A/CoreVideo (compatibility version 1.2.0, current version 1.5.0)
	/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 1226.0.0)
	/usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.12)
	/System/Library/Frameworks/CoreAudio.framework/Versions/A/CoreAudio (compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox (compatibility version 1.0.0, current version 1000.0.0)
	/System/Library/Frameworks/CoreHaptics.framework/Versions/A/CoreHaptics (compatibility version 1.0.0, current version 1.0.0, weak)
	/System/Library/Frameworks/GameController.framework/Versions/A/GameController (compatibility version 1.0.0, current version 12.1.3, weak)
	/System/Library/Frameworks/ForceFeedback.framework/Versions/A/ForceFeedback (compatibility version 1.0.0, current version 1.0.2)
	/System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 170.0.0)
	/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
	/System/Library/Frameworks/Metal.framework/Versions/A/Metal (compatibility version 1.0.0, current version 367.4.0, weak)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
	/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 61439.41.1)
	/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit (compatibility version 45.0.0, current version 2575.20.19)
	/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 1883.1.10)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 3107.0.0)

Given Homebrew reports those as system libraries, I think we're OK, this is probably as static as it gets.

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2024

I updated the PR.

Please test the following binaries generated by GitHub Actions:

@muvaf
Copy link
Contributor

muvaf commented Nov 23, 2024

@rom1v

macOS didn't run at first, asking it to be explicitly allowed in the settings, which I did, then it worked with following logs:

 /tmp/scrcpy-macos-test ------------------------------------------------------------------------------------------------------------- 08:34:14 PM
> ls
adb           icon.png      scrcpy        scrcpy-server scrcpy_bin

 /tmp/scrcpy-macos-test --------------------------------------------------------------------------------------------------- KILL  5s  08:34:25 PM
> ./scrcpy
scrcpy 2.7 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     --> (tcpip)  emulator-5554                   device  sdk_gphone64_arm64
./scrcpy-server: 1 file pushed, 0 skipped. 218.7 MB/s (90228 bytes in 0.000s)
[server] INFO: Device: [Google] google sdk_gphone64_arm64 (Android 15)
ERROR: Could not get icon path
WARN: Could not load icon
INFO: Renderer: metal
INFO: Texture: 1080x2400
2024-11-23 20:34:48.398 scrcpy_bin[88997:1891324] +[IMKClient subclass]: chose IMKClient_Modern
2024-11-23 20:34:48.398 scrcpy_bin[88997:1891324] +[IMKInputSession subclass]: chose IMKInputSession_Modern

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2024

macOS didn't run at first, asking it to be explicitly allowed in the settings

OK, it's probably a security on macOS when running a downloaded binary?

Do you know why these logs:

2024-11-23 20:34:48.398 scrcpy_bin[88997:1891324] +[IMKClient subclass]: chose IMKClient_Modern
2024-11-23 20:34:48.398 scrcpy_bin[88997:1891324] +[IMKInputSession subclass]: chose IMKInputSession_Modern

?

Do you have them if you build scrcpy yourself (dynamically)?

@muvaf
Copy link
Contributor

muvaf commented Nov 23, 2024

This is what I see on Linux amd64:

/tmp/scrcpy-linux-test                                                                                 08:36:40 PM
❯ ./scrcpy
scrcpy 2.7 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     --> (tcpip)  emulator-5554                   device  sdk_gphone64_x86_64
./scrcpy-server: 1 file pushed, 0 skipped. 225.0 MB/s (90228 bytes in 0.000s)
[server] INFO: Device: [Google] google sdk_gphone64_x86_64 (Android 15)
[server] INFO: Retrying with -m2560...
[server] ERROR: Encoding error: java.lang.IllegalArgumentException: 
libEGL warning: egl: failed to create dri2 screen
[server] WARN: Could not get initial audio timestamp
INFO: Renderer: opengles2
INFO: OpenGL version: OpenGL ES 3.2 Mesa 24.0.9-0ubuntu0.2
INFO: Trilinear filtering enabled
INFO: Texture: 1344x2992
INFO: Texture: 1152x2560

Note that in the first archive you provided in PR description, I was seeing NVIDIA:

/tmp/scrcpy-linux-ga                                                                                   08:37:44 PM
❯ ./scrcpy
scrcpy 2.7 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO:     --> (tcpip)  emulator-5554                   device  sdk_gphone64_x86_64
./scrcpy-server: 1 file pushed, 0 skipped. 104.1 MB/s (90228 bytes in 0.001s)
[server] INFO: Device: [Google] google sdk_gphone64_x86_64 (Android 15)
[server] ERROR: Encoding error: java.lang.IllegalArgumentException: 
[server] INFO: Retrying with -m2560...
[server] WARN: Could not get initial audio timestamp
INFO: Renderer: opengl
INFO: OpenGL version: 4.6.0 NVIDIA 560.35.03
INFO: Trilinear filtering enabled
INFO: Texture: 1344x2992
INFO: Texture: 1152x2560

Do you have them if you build scrcpy yourself (dynamically)?

I haven't built scrcpy myself dynamically

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2024

ERROR: Could not get icon path
WARN: Could not load icon

Fixed by:

diff --git app/data/scrcpy_static_wrapper.sh app/data/scrcpy_static_wrapper.sh
index 4dcce9a72..56243471f 100755
--- app/data/scrcpy_static_wrapper.sh
+++ app/data/scrcpy_static_wrapper.sh
@@ -2,5 +2,5 @@
 cd "$(dirname ${BASH_SOURCE[0]})"
 export ADB=./adb
 export SCRCPY_SERVER_PATH=./scrcpy-server
-export ICON=./icon.png
+export SCRCPY_ICON_PATH=./icon.png
 ./scrcpy_bin "$@"

@muvaf
Copy link
Contributor

muvaf commented Nov 23, 2024

@rom1v I'm seeing those log lines even when I use scrcpy from brew install scrcpy

@muvaf
Copy link
Contributor

muvaf commented Nov 23, 2024

@rom1v Will you use these scripts only to publish to Github releases or are you planning to use them in Homebrew formula as well?

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2024

Note that in the first archive you provided in PR description, I was seeing NVIDIA:

In the initial version, in the build-linux job, I used:

          sudo apt install -y meson ninja-build nasm ffmpeg libsdl2-2.0-0 \
             libsdl2-dev libavcodec-dev libavdevice-dev libavformat-dev \
             libavutil-dev libswresample-dev libusb-1.0-0 libusb-1.0-0-dev

Now, I use:

          sudo apt update
          sudo apt install -y meson ninja-build nasm libudev-dev \
            libasound2-dev libpulse-dev

(I added libasound2-dev and libpulse-dev otherwise I had no sound)

I think I will revert that change, many feature will be missing I guess when building without the SDL and FFmpeg dependencies.

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2024

Will you use these scripts only to publish to Github releases or are you planning to use them in Homebrew formula as well?

Only for official releases.

@muvaf
Copy link
Contributor

muvaf commented Nov 23, 2024

Are you planning on adding linux-arm64 and darwin-amd64 architectures as well potentially via cross-compilation?

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2024

Are you planning on adding linux-arm64 and darwin-amd64 architectures as well potentially via cross-compilation?

It would be great.

However, I won't work on it soon, my goal for now is just to have at least "experimental" static linux and macos releases for "the general use case" before publishing scrcpy 3.0 (with virtual displays and opengl filters).

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2024

Here are new binaries, with the known problems fixed:

Please test 🙂

rom1v added a commit to rom1v/scrcpy that referenced this pull request Nov 24, 2024
Provide a prebuilt binary for macOS.

PR Genymobile#5515 <Genymobile#5515>

Co-authored-by: Muvaffak Onus <me@muvaf.com>
rom1v added a commit to rom1v/scrcpy that referenced this pull request Nov 24, 2024
Build the server without gradle to make sure that the script works.

PR Genymobile#5515 <Genymobile#5515>
rom1v and others added 16 commits November 24, 2024 15:40
Since commit 2687d20, the Makefile
named release.mk stopped handling dependencies between recipes, because
they have to be executed separately (from different Github Actions
jobs).

Using a Makefile no longer provides any real benefit. Replace it by
several individual release scripts for simplicity and readability.

Refs #5306 <#5306>
PR #5515 <#5515>
Use a separate GitHub Action job to build and test the server.

PR #5515 <#5515>
Create separate jobs for packaging win32 and win64 releases.

PR #5515 <#5515>
The upload-artifact action does not preserve file permissions:
<https://github.com/actions/upload-artifact?#permission-loss>

Even if it is not critical for Windows releases, it will be for other
platforms. Wrap everything in a tarball to keep original permissions.

PR #5515 <#5515>
All --disable, then all --enable.

PR #5515 <#5515>
They are not used, and this prevents Linux builds from working if the
dependencies are unavailable.

PR #5515 <#5515>
This will make it easy to conditionally add items.

PR #5515 <#5515>
Extract the code that processes arguments into a function.

This will make it optional, so the script that only downloads the
official ADB binaries will not use arguments.

PR #5515 <#5515>
This will allow adding similar scripts for other platforms.

PR #5515 <#5515>
Make dependencies build scripts more flexible, to accept a build type
(native or cross) and a link type (static or shared).

This lays the groundwork for building binaries for Linux and macOS.

PR #5515 <#5515>
Use static dependencies if the option is set.

PR #5515 <#5515>
Make package_client.sh accept an archive format.

PR #5515 <#5515>
Provide a prebuilt binary for Linux.

Fixes #5327 <#5327>
PR #5515 <#5515>
The command sha256sum does not exist on macOS, but `shasum -a256` works
both on Linux and macOS.

PR #5515 <#5515>

Co-authored-by: Romain Vimont <rom@rom1v.com>
Signed-off-by: Romain Vimont <rom@rom1v.com>
Provide a prebuilt binary for macOS.

Fixes #1733 <#1733>
Fixes #3235 <#3235>
Fixes #4489 <#4489>
Fixes #5327 <#5327>
PR #5515 <#5515>

Co-authored-by: Muvaffak Onus <me@muvaf.com>
Build the server without gradle to make sure that the script works.

PR #5515 <#5515>
@rom1v rom1v merged commit 6f9520f into dev Nov 24, 2024
@Genxster1998
Copy link
Contributor

@rom1v I have used macos-13 here Genxster1998@db01900

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 24, 2024

@Genxster1998 Could you please re-implement this on top of current dev and open a PR, please? (with minimal changes compared to the macos-latest job)

@muvaf
Copy link
Contributor

muvaf commented Nov 24, 2024

@rom1v FYI, I tested released tarballs in both Linux amd64 and macOS arm64 and they both worked just as expected. Thank you for getting the scripts to the finish line so that we get the static builds so soon!

@Genxster1998
Copy link
Contributor

@rom1v I don't have much idea about cross compiling with universal binary as target. It would just need separate build config for Intel and arm64. And ofcourse renaming release because current macos release in not universal.

@LeeBinder
Copy link

LeeBinder commented Nov 24, 2024

@rom1v I have used macos-13 here Genxster1998@db01900

I can confirm your latest Intel binary from Actions build-macos-x86_64-intermediate also running just fine in the latest macOS Monterey 12.7.6

@Genxster1998
Copy link
Contributor

Genxster1998 commented Nov 24, 2024

@LeeBinder 👍 I am encountering you a lot , in hacintosh repos , sometime in kernelsu or zygisk stuffs.lol 🤣 😜

@LeeBinder
Copy link

..maybe I'm your cyberspace twin brother 🤣 😜

@Genxster1998
Copy link
Contributor

@LeeBinder Can you see if https://github.com/Genxster1998/scrcpy/actions/runs/12015099730 works without a local wrapper script , just icon.png and scrcpy-server in same dir and adb installed.

@LeeBinder
Copy link

YES, you DID it! Executing scrcpy_bin has the desired result, with the icon in the dock - respect!

BTW, in the info for scrcpy_bin it says "requires macOS 13", but it runs fine even in macOS 12 Monterey (maybe even below).

@rom1v
Copy link
Collaborator Author

rom1v commented Dec 9, 2024

Please test #5644 release builds with scrcpy --video-codec=av1 (with a phone having an AV1 encoder).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants