[ COMMON ] Build Tools Problems

[ChthonVII]

There are a number of problems with the build tools, mostly vcpkg. It took me hours to troubleshoot all the issues created or revived by the last dependency update bumping vpkg and cmake and to get back to the point where I could build again. This matters because it's such a headache that it strongly discourages people from contributing to this project. If I was a brand new potential contributor, I would have just walked away after a couple hours.

1. CMakePresets.json declares that it is version 6. But the latest Visual Studio 2022 only supports versions 2 through 5. The file isn't using anything exclusive to version 6, however, and it works if changed to 5. It should be changed to 5.
2. By default, cmake uses the SDK matching the running version of Windows. This is a problem for me, since I'm using an old Win8 laptop to compile. (My daily-use PC is Linux.) In the future it will become a problem for many people as Win10 and Win11 diverge. The solution is to expressly set the SDK to use in CMakeLists.txt. Placing set(CMAKE_SYSTEM_VERSION 10.0.20348 CACHE TYPE INTERNAL FORCE) anywhere before the first appearance of project causes that version of the Win10 SDK to be used. (This might not be the best version to use; it's just what I had installed.)
3. vcpkg wants to download grep-3.0-2-x86_64.pkg.tar.xz as a build tool for several dependencies, but that version is outdated and has been removed from all the mirrors that vcpkg checks.
   • Here is the pull request for vcpkg to fix it, but vcpkg is such a shitshow that its author is having trouble getting it merged. (Also, it sounds like this sort of thing is an ongoing problem affecting more than just this one package.)
   • As per the discussion in that pull request, there still exists one mirror that hasn't removed it yet. (It's just not one that vcpkg checks.)
   • As a workaround, you can download the file from that mirror, then place a copy in C:\vcpkg\downloads, then rename it to msys-grep-3.0-2-x86_64.pkg.tar.xz, then try to build. However, this is needed for several dependencies of FFNx, and vcpkg consumes it without caching each time it uses it, causing the build process to fail on the next dependency when it tries to download it again and can't. You can eventually get all the dependencies built by repeatedly placing the file in C:\vcpkg\downloads, building one thing, failing on the next one, and starting over.
   • A better workaround would be to find vcpkg's list of mirrors and edit it to include the one mirror that still has this file.
   • This workaround should be noted in FFNx's readme, at least until vcpkg gets their act together.
4. vcpkg wants to download ogg-v1.3.5.tar.gz. But vcpkg uses winhttp to establish TLS connections. On older versions of Windows, winhttp doesn't support TLS 1.3, but the xiph.org server doesn't support anything below version 1.3, making the download impossible. The workaround is to download the file in a browser, then place in C:\vcpkg\downloads, then rename it to xiph-ogg-v1.3.5.tar.gz, then try to build. This workaround should be noted in FFNx's readme.
5. Speex fails to build with a permissions error if the directory C:\vcpkg\downloads\temp does not exist. Sometimes (I'm not sure why or when) vcpkg will do some clean up and delete that directory, causing the build to fail. The workaround is to create the directory manually, then try again. This workaround should be noted in FFNx's readme.

Overall, this exercise has left me with a very firm impression that vcpkg is a flaming pile of dogshit. I'd strongly urge migrating to something else. Anything else.

Please don't bump the vcpkg requirement unless it's necessary. Dealing with this bump was a lot of pain, and I fully expect every bump to involve things going horribly wrong with vcpkg.

[julianxhokaxhiu]

Thanks a lot for feedback, as much I understand your frustration assuming the entire setup is wrong without context is actually far from the reality. But let me try to better explain myself by trying to answer you on each point:

1. Actually the latest version of CMake, as Visual Studio 2022 17.6.2 Community, is:

   ❯ cmake --version
   cmake version 3.26.0-msvc3
   CMake suite maintained and supported by Kitware (kitware.com/cmake)

   This means you can make fully use of the repo as it is. No need to change anything.
2. By default CMake uses the latest SDK of Windows you have installed, whatever it is. No need to enforce anything, so it works with past, present and future installations :)
3. I understand your frustrations with vcpkg, but I'm always in the frontline reporting and even fixing issues with them when they happen on FFNx. The trick is to be always on the latest master clone, and run FFNx on top of that. All the rest will be automatically handled for you.
4. Same as above, it's "package manager" stuff, I don't see why I should be involved in documenting workarounds for vcpkg when this is not happening here or also on the CI so to say. And if it happens, as already said I'm usually in the frontline trying to find a workaround and/or a solution for everyone contributing.
5. Also this is vcpkg stuff, and the story goes the same as above.

I'm sorry you got so frustrated with it, but one potential solution I can suggest you is to migrate to Windows 10 or 11 which are actually the officially supported solutions by Microsoft themselves. If you wish to keep building on any other Windows version is at your own risk. Addittionally make sure you follow all the steps described in the Readme from installation to configuration. If something is not working on top of those let me know and I'm more than happy to amend with better steps or missing ones, if any.

Finally, when I bump it, is only when vcpkg releases a new official one which helps me also avoiding exactly all the issues you mentioned above, aside of the fact that I like to keep the project dependencies fresh and reduces for me the chances to have to support old code. Please remember no one is paid to keep this project running, including myself, nor I've ever a penny for it.

I'm open for feedback and I'm happy to integrate anything that comes up, but the last of the things I want to be told is what to do, and what not, when there are no real arguments on the table.

Happy to help if there's anything real I can do.

[ChthonVII]

1. Actually the latest version of CMake, as Visual Studio 2022 17.6.2 Community, is:

I'm not talking about the version of cmake. I'm talking about the version of CMakePresets.json itself. Right on line 2 it says "version": 6,. But Visual Studio doesn't support version 6. See docs. So the config fails right off the bat. It needs to be 5.

2. By default CMake uses the latest SDK of Windows you have installed, whatever it is. No need to enforce anything, so it works with past, present and future installations :)

From the docs:

"When the CMAKE_SYSTEM_NAME variable takes its default value then CMAKE_SYSTEM_VERSION is by default set to the same value as the CMAKE_HOST_SYSTEM_VERSION variable so that the build targets the host system version."

Though it looks like this will be replaced with (hopefully) smarter autodetection in cmake 3.4. So maybe this won't be a problem anymore by the time Win10 and Win11 SDKs diverge.

vcpkg... The trick is to be always on the latest master clone, and run FFNx on top of that. All the rest will be automatically handled for you... I don't see why I should be involved in documenting workarounds for vcpkg when this is not happening here...

These problems ARE happening here! Why the &$%#^@ else would I be reporting it? If a brand new contributor using Win10/11 downloaded fresh copies of Visual Studio and vcpkg TODAY and tried to build FFNx, they would definitely encounter problems 1 and 3, plus maybe encounter problem 5 at random. (If they were using Win8.1, they'd also definitely encounter problems 2 and 4.) And they would not be able to build. Not being able to build FFNx using the directions in the readme is a terrible experience for a potential new contributor, and it's going to drive people away. The least we can do is document the workarounds so that people can actually build FFNx without spending hours troubleshooting vcpkg.

(The only reason you are not encountering problem 3 is that you got your dependencies built before the deprecated version of grep that vcpkg wants to download was taken down. Try deleting your vcpkg install and asset cache and starting over fresh, and I guarantee you will hit this problem and won't be able to build FFNx without the workaround here, or until this finally gets fixed on the vcpkg side.)

one potential solution I can suggest you is to migrate to Windows 10 or 11

First of all, no, I'm not going to spend money on an operating system I actively loathe for the sake of a hobby project.
Second, switching to Win10/11 would only fix problems 2 and 4. Problems 1, 3, and 5 exist independent of the OS version.