Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add native arm64 build slice to all Windows dependencies #190

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

PatTheMav
Copy link
Member

@PatTheMav PatTheMav commented Jun 5, 2023

Description

This is an exploratory PR for possible future native ARM64 builds on Windows.

Some dependencies cannot be built for ARM64 yet, others require specific workarounds, yet others can be built for ARM64 but themselves are too recent to be supported by OBS Studio.

This PR will stay in draft state until these and other issues are solved and are in a state that do not result in unnecessary support burden to current maintainers and supporters.

This PR is based on #186 and the PRs it depends on.

Motivation and Context

Due to Visual Studio 17 2022 having full support for ARM64 builds as well as clang- and Ninja-based builds, selecting the correct build architecture for MSVC builds does most of the work to achieve native builds, there are some caveats though:

  • libvpx requires disabling more advanced NEON features not made available by Microsoft's ARM64 implementation.
  • x264 requires an update of the gas-preprocessor it ships with - luckily FFmpeg provides a very recent version of it (the version shipped with x264 is 9 years old)
  • FFmpeg requires the same gas-processor to be made available in the PATH
  • svt-av1 has not been made available as it's not well optimised for ARM64 - dav1d would be a better alternative, but aom is "good enough" for now
  • Qt6 builds without any additional hacks or issues
  • LuaJIT has merged ARM64 and is available now
  • Python supports Windows ARM64 since version 3.11, alas OBS Studio does not support Python 3.11 at runtime yet (you could build OBS Studio with it, but people couldn't add its libraries at runtime)
  • mbedtls is still broken on ARM64 with version 3.5.1
  • libdatachannel requires at least mbedtls 3.4.0, so both are entirely broken on ARM64 right now
  • vpl is disabled for ARM64
  • Aja NTV2 is disabled for ARM64

How Has This Been Tested?

Tested on Windows 11 ARM64.

Types of changes

  • New feature (non-breaking change which adds functionality)

Checklist:

  • My code has been run through clang-format.
  • I have read the contributing document.
  • My code is not on the master branch.
  • The code has been tested.
  • All commit messages are properly formatted and commits squashed where appropriate.
  • I have included updates to all appropriate documentation.

@driver1998
Copy link

What is the error exactly in building mbedtls? I saw upstream fixed similar issues multiple times.

@PatTheMav
Copy link
Member Author

What is the error exactly in building mbedtls? I saw upstream fixed similar issues multiple times.

Compiling aes.c breaks with version 3.5.1:

error C1189: #error:  This header is specific to X86, X64, ARM64, and ARM64EC targets

This is possibly fixed by Mbed-TLS/mbedtls#8168, so we're waiting for MbedTLS to cut a new release with those fixes included.

@PatTheMav PatTheMav force-pushed the windows-arm64-native branch 6 times, most recently from 6f09be1 to 58c4660 Compare January 13, 2024 00:28
@tommyvct
Copy link
Contributor

This needs to be rebased and merge #227 in.

@RytoEX
Copy link
Member

RytoEX commented Jan 17, 2024

This needs to be rebased and merge #227 in.

@PatTheMav and I work closely together off-thread. Rest assured, that we discuss this regularly. However, do note that this is being worked on as time and other priorities allow. Thank you for your understanding.

@tommyvct
Copy link
Contributor

together off-thread. Rest assured, that we discuss this regularly.

May I join the conversation?

@RytoEX
Copy link
Member

RytoEX commented Jan 17, 2024

together off-thread. Rest assured, that we discuss this regularly.

May I join the conversation?

I appreciate the enthusiasm, but these are conversations that we have either on voice calls or in ongoing DMs about getting stuff to work. It is mostly stuff like "so about that PR" and "here's an idea" and "here's how we might get that to work when we have time". The point is that he doesn't need a reminder to rebase this. He's aware. This is on our roadmap to address.

@tommyvct
Copy link
Contributor

How about we start a separate thread in the development channel? Or I could just ask and @ both of you in the channel?

Copy link
Member

@RytoEX RytoEX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some quick observations. I did get a local Qt build working, but it requires setting both QT_QMAKE_TARGET_MKSPEC and QT_HOST_PATH. I've commented how I might approach this for CI. We can attempt to optimize it later.

deps.qt/qt6.ps1 Outdated Show resolved Hide resolved
deps.windows/60-python.ps1 Outdated Show resolved Hide resolved
deps.windows/40-detours.ps1 Outdated Show resolved Hide resolved
deps.windows/40-luajit.ps1 Outdated Show resolved Hide resolved
utils.pwsh/Setup-Target.ps1 Show resolved Hide resolved
deps.ffmpeg/99-ffmpeg.ps1 Outdated Show resolved Hide resolved
deps.ffmpeg/30-libvpx.ps1 Outdated Show resolved Hide resolved
@tommyvct tommyvct mentioned this pull request Mar 12, 2024
6 tasks
@RytoEX
Copy link
Member

RytoEX commented Apr 26, 2024

Was just skimming back over this.

  • x264 requires an update of the gas-preprocessor it ships with - luckily FFmpeg provides a very recent version of it (the version shipped with x264 is 9 years old)
  • FFmpeg requires the same gas-processor to be made available in the PATH

This seems like it should be a trivial Merge Request on the x264 repo. Ideally, I'd like the change to happen there at some point.

  • svt-av1 has not been made available as it's not well optimised for ARM64 - dav1d would be a better alternative, but aom is "good enough" for now

VideoLAN's dav1d is a decoder only. It is not an encoder. Xiph's rav1e is an encoder.

@PatTheMav PatTheMav force-pushed the windows-arm64-native branch 6 times, most recently from 3f2d466 to bc8100a Compare May 29, 2024 16:51
@tommyvct
Copy link
Contributor

The next major part is updating obs-studio's build system to support ARM64 on Windows, which requires some extensive rewrites as the build system implicitly only knows Windows as an AMD64 platform, which also requires support for 32-bit x86 applications (e.g. our capture hook needs to be built and shipped as x86 as well to support capturing older games).

This is specific to Game Capture, right? I guess for now we could just let this slide for now, since the last known good ARM64 build works perfectly fine on Desktop capture, and that would cover at least 90% of the use cases. Snapdragon X Elite/Plus isn't intended to be a video game platform, so this could be put at the bottom of the list.

@RytoEX
Copy link
Member

RytoEX commented Jun 13, 2024

The next major part is updating obs-studio's build system to support ARM64 on Windows, which requires some extensive rewrites as the build system implicitly only knows Windows as an AMD64 platform, which also requires support for 32-bit x86 applications (e.g. our capture hook needs to be built and shipped as x86 as well to support capturing older games).

This is specific to Game Capture, right? I guess for now we could just let this slide for now, since the last known good ARM64 build works perfectly fine on Desktop capture, and that would cover at least 90% of the use cases. Snapdragon X Elite/Plus isn't intended to be a video game platform, so this could be put at the bottom of the list.

Game Capture and Virtual Camera both require both 64-bit and 32-bit components on x64 Windows. The minute we ship a build without features we'll be asked why it doesn't include X, Y, and Z repeatedly. Given that historically, we have only had to contend with x86 (32-bit) and x64 (64-bit) on Windows, there are components that include 32 or 64 or the like in their name, so the build specifications for these (in CMake) and code that interfaces with those filenames must be made aware that a non-32-bit, non-64-bit type/name is possible (arm).

Rest assured, we have considered these points when thinking about our path forward.

@tommyvct
Copy link
Contributor

Well, we could release a alpha version that lacks that feature, put a huge dialog with huge font, that is non-dismissible for 30 seconds, saying that by continue using you agree to acknowledge that feature X, Y, and Z are missing as they are being worked on, and please don't ask about these features anywhere.

The problem I see was Qualcomm used OBS Studio to promote their Snapdragon X Elite platform, while we have no support for Windows on ARM at all, and absolutely no contribution from Qualcomm themselves.

@RytoEX
Copy link
Member

RytoEX commented Jun 13, 2024

Well, we could release a alpha version that lacks that feature, put a huge dialog with huge font, that is non-dismissible for 30 seconds, saying that by continue using you agree to acknowledge that feature X, Y, and Z are missing as they are being worked on, and please don't ask about these features anywhere.

Thanks for the suggestion, but we already have our own plans for the path forward.

The problem I see was Qualcomm used OBS Studio to promote their Snapdragon X Elite platform, while we have no support for Windows on ARM at all, and absolutely no contribution from Qualcomm themselves.

We're aware.

@RytoEX
Copy link
Member

RytoEX commented Jun 13, 2024

This is getting noisy, and I would really like to keep comments here to reviewing the code changes rather than policy, thanks.

Copy link
Member

@RytoEX RytoEX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nits.

Drop the Sparkle commit. Sparkle is already updated in this repo.

Would still like to see these additional patches submitted to the dependency repos to see if they'll adopt them.

deps.qt/qt6.ps1 Outdated Show resolved Hide resolved
deps.qt/qt6.ps1 Outdated Show resolved Hide resolved
deps.qt/qt6.ps1 Outdated Show resolved Hide resolved
deps.qt/qt6.ps1 Outdated Show resolved Hide resolved
@jagp-qcom
Copy link

Qualcomm is working on ARM native porting. Very soon we shall be more active here.

@Fenrirthviti
Copy link
Member

Qualcomm is working on ARM native porting. Very soon we shall be more active here.

Qualcomm has not reached out to us in any capacity at this time. We hope that they would contact us if they are starting work on the project to port it to a new platform to ensure that efforts are aligned.

@DoM1niC
Copy link

DoM1niC commented Jul 10, 2024

I got the new Surface X, if somebody is needed for Tests so feel free to link me a Build 🙂👍🏻

@sameeraman
Copy link

I'm happy to test also if help needed on that. Keen to get a ARM64 windows version of the OBS.

@RytoEX
Copy link
Member

RytoEX commented Jul 25, 2024

Any test builds would likely be provided on PRs on the obs-studio repo. In the meantime, I would like to keep comments focused on review or feedback from people who have actually tried to build/test these changes. Thank you for understanding.

@talynone
Copy link

talynone commented Aug 5, 2024

As Windows ARM64 supports running x64 as well as x86 binaries (ARM32 as well, but we're not concerned about those), it effectively means that users might be running apps and games with those architectures as well. Thus an ARM64 build needs x64 and x86 variants of the capture hooks and virtual camera, which adds complexity.

Just a minor correction, Windows 11 ARM edition does not support ARM32 executables on the new Snapdragon Elite processors, and there's no emulation layer for it either, they simply won't work. It's the reason Android subsystem for Windows broke on the Elite processors, it relied on ARM32 binaries.

More info here: https://learn.microsoft.com/en-us/windows/arm/arm32-to-arm64

@PatTheMav PatTheMav force-pushed the windows-arm64-native branch 4 times, most recently from 210ffa9 to 8a8746b Compare August 5, 2024 22:07
@RytoEX RytoEX self-assigned this Aug 21, 2024
@PatTheMav PatTheMav marked this pull request as ready for review August 21, 2024 17:58
@RytoEX
Copy link
Member

RytoEX commented Aug 21, 2024

I don't think we can update to libsrt 1.5.3 or we'll run into the issues linked in #240.

Restating that, if possible, I'd like to see the patches for SRT and libpng upstreamed to their respective repos. I'm mostly only concerned that the patches get submitted. I have been unable to register an account on the VideoLAN GitLab instance to open an Issue about their outdated gas-preprocessor.

@RytoEX
Copy link
Member

RytoEX commented Sep 1, 2024

As discussed off-thread with @PatTheMav , updating libsrt to 1.5.3 has known issues in OBS Studio on Windows. These are supposedly resolved in libsrt 1.5.4, which is not yet out/stable. It's not clear to me that libsrt 1.5.3+ is required for WoA/arm64. Can we confirm?

Separately, MbedTLS 3.6.0+ is required for WoA/arm64, as far as I know. However, we cannot update that until Haivision/srt#2957 is fixed. This seems to have been addressed by deps.ffmpeg/patches/srt/0002-update-mbedtls-discovery-windows.patch included in this PR. We should submit that as a PR to the srt repo. They can decide whether or not to merge it.

If we need libsrt 1.5.3+, either we wait for all dependencies to be stable, or we teach the Windows dependency build scripts to build different versions per target architecture.

Additionally, I still cannot build obs-studio on Windows against libsrt 1.5.3+ and mbedTLS 3.6.0+ due to the bcrypt linking issue, even with the patch here. I haven't tried obsproject/obs-studio#11056 because it fails on CI, and needs to be rebased (I'm guessing gersemi changes).

@PatTheMav PatTheMav force-pushed the windows-arm64-native branch 2 times, most recently from f224dee to 98eb7ab Compare September 17, 2024 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.