[BUG] r22 appears to break cmake compiler and OS detection on Windows #1427
Assignees
Labels
Comments
|
Just a note regarding the command above. I do know that r22 doesn't have support for API16 (at least the runtime libs for it are absent, clang wrapper scripts are still there), but it's not relevant in this case - the error appears to be unrelated at least. However, for API21 build |
|
Looks like CMake hasn't set the Our QA will have tested Windows before we shipped though. We probably need a repro case here.
|
|
Assigning to hhb in case he has any ideas. |
It seems as if ARM support wasn't there at all, note this error: And look at the list of supported CPUs, it contains only x86.
My bad, looked in a wrong directory (aarch64) :)
|
|
I think the However, I added the following snippet to our if(WIN32)
message(STATUS "WIN32 set")
endif()
if(MINGW)
message(STATUS "MINGW set")
endif()And the output is: |
|
It seems cmake 3.19.2 could detect the right platform. That's my temporary workaround. |
You probably are using cmake's NDK support instead of NDK toolchain file. In that case CMake >= 3.19 is required for NDK r22. (I'm investing why MINGW is used there in this issue |
|
Seems it is not trivial to build Xamarin.Android. Do you have a smaller repro case? Also can you try to copy r21 |
You don't really need to build it all. On Windows (from VS prompt), after cloning and restoring submodules, you need the following steps:
I don't use Windows locally, but I'll try to test that ASAP. |
The problem seems fixed after replacing |
|
@fuchg3 thanks, you saved me from a few hours of installing software :) |
I've just tested it on CI and it does fix the detection |
grendello
added a commit
to grendello/xamarin-android-tools
that referenced
this issue
Jan 20, 2021
Context: dotnet/android#5499 Context: dotnet/android#5526 Context: android/ndk#1427 Context: https://developer.android.com/studio/command-line/variables#envar Xamarin.Android is not (yet) compatible with the recently released Android NDK r22 version. Azure build images have recently rolled out an update which includes NDK r22 and, thus, it breaks builds for customers using any form of Xamarin.Android AOT build. In attempt to detect broken/incompatible NDK versions as well as select the "best one", this commit adds code to scan the known NDK locations in search of the preferred version. The search is conducted as follows: 1. If the user selected a preferred NDK location, it is always used. 2. Locations specified in the `ANDROID_{HOME,SDK_ROOT}` environment variables are returned next. 3. Directories in the `PATH` environment variable are examined to find a valid NDK location. 4. OS-specific known NDK locations are considered. For each of the returned locations, we now look for the Android SDK packages containing the NDK. There are two kinds of such packages: * `ndk-bundle` is the older package which allows for installation of only one NDK inside the SDK directory * `ndk/*` is a newer package which allows for installation of several NDK versions in parallel. Each subdirectory of `ndk` is an `X.Y.Z` version number of the NDK. In each of these directories we look for the `source.properties` file from which we then extract the NDK version and then we sort thus discovered NDK instances using their version as the key, in the descending order. The latest compatible (currently: less than 22 and more than 15) version is selected and its path returned to the caller.
grendello
added a commit
to grendello/xamarin-android-tools
that referenced
this issue
Jan 20, 2021
Context: dotnet/android#5499 Context: dotnet/android#5526 Context: android/ndk#1427 Context: https://developer.android.com/studio/command-line/variables#envar Xamarin.Android is not (yet) compatible with the recently released Android NDK r22 version. Azure build images have recently rolled out an update which includes NDK r22 and, thus, it breaks builds for customers using any form of Xamarin.Android AOT build. In attempt to detect broken/incompatible NDK versions as well as select the "best one", this commit adds code to scan the known NDK locations in search of the preferred version. The search is conducted as follows: 1. If the user selected a preferred NDK location, it is always used. 2. Locations specified in the `ANDROID_{HOME,SDK_ROOT}` environment variables are returned next. 3. Directories in the `PATH` environment variable are examined to find a valid NDK location. 4. OS-specific known NDK locations are considered. For each of the returned locations, we now look for the Android SDK packages containing the NDK. There are two kinds of such packages: * `ndk-bundle` is the older package which allows for installation of only one NDK inside the SDK directory * `ndk/*` is a newer package which allows for installation of several NDK versions in parallel. Each subdirectory of `ndk` is an `X.Y.Z` version number of the NDK. In each of these directories we look for the `source.properties` file from which we then extract the NDK version and then we sort thus discovered NDK instances using their version as the key, in the descending order. The latest compatible (currently: less than 22 and more than 15) version is selected and its path returned to the caller.
grendello
added a commit
to grendello/xamarin-android-tools
that referenced
this issue
Jan 20, 2021
Context: dotnet/android#5499 Context: dotnet/android#5526 Context: android/ndk#1427 Context: https://developer.android.com/studio/command-line/variables#envar Xamarin.Android is not (yet) compatible with the recently released Android NDK r22 version. Azure build images have recently rolled out an update which includes NDK r22 and, thus, it breaks builds for customers using any form of Xamarin.Android AOT build. In attempt to detect broken/incompatible NDK versions as well as select the "best one", this commit adds code to scan the known NDK locations in search of the preferred version. The search is conducted as follows: 1. If the user selected a preferred NDK location, it is always used. 2. Locations specified in the `ANDROID_{HOME,SDK_ROOT}` environment variables are returned next. 3. Directories in the `PATH` environment variable are examined to find a valid NDK location. 4. OS-specific known NDK locations are considered. For each of the returned locations, we now look for the Android SDK packages containing the NDK. There are two kinds of such packages: * `ndk-bundle` is the older package which allows for installation of only one NDK inside the SDK directory * `ndk/*` is a newer package which allows for installation of several NDK versions in parallel. Each subdirectory of `ndk` is an `X.Y.Z` version number of the NDK. In each of these directories we look for the `source.properties` file from which we then extract the NDK version and then we sort thus discovered NDK instances using their version as the key, in the descending order. The latest compatible (currently: less than 22 and more than 15) version is selected and its path returned to the caller.
grendello
added a commit
to grendello/xamarin-android-tools
that referenced
this issue
Jan 20, 2021
Context: dotnet/android#5499 Context: dotnet/android#5526 Context: android/ndk#1427 Context: https://developer.android.com/studio/command-line/variables#envar Xamarin.Android is not (yet) compatible with the recently released Android NDK r22 version. Azure build images have recently rolled out an update which includes NDK r22 and, thus, it breaks builds for customers using any form of Xamarin.Android AOT build. In attempt to detect broken/incompatible NDK versions as well as select the "best one", this commit adds code to scan the known NDK locations in search of the preferred version. The search is conducted as follows: 1. If the user selected a preferred NDK location, it is always used. 2. Locations specified in the `ANDROID_{HOME,SDK_ROOT}` environment variables are returned next. 3. Directories in the `PATH` environment variable are examined to find a valid NDK location. 4. OS-specific known NDK locations are considered. For each of the returned locations, we now look for the Android SDK packages containing the NDK. There are two kinds of such packages: * `ndk-bundle` is the older package which allows for installation of only one NDK inside the SDK directory * `ndk/*` is a newer package which allows for installation of several NDK versions in parallel. Each subdirectory of `ndk` is an `X.Y.Z` version number of the NDK. In each of these directories we look for the `source.properties` file from which we then extract the NDK version and then we sort thus discovered NDK instances using their version as the key, in the descending order. The latest compatible (currently: less than 22 and more than 15) version is selected and its path returned to the caller.
jonpryor
pushed a commit
to dotnet/android-tools
that referenced
this issue
Jan 20, 2021
Context: actions/runner-images#2420 Context: dotnet/android#5499 Context: dotnet/android#5526 Context: android/ndk#1427 Context: https://developer.android.com/studio/command-line/variables#envar Xamarin.Android is not (yet) compatible with the recently released Android NDK r22 version. Azure build images have recently rolled out an update which includes NDK r22 and, thus, it breaks builds for customers using any form of Xamarin.Android AOT build. In an attempt to detect broken/incompatible NDK versions as well as to select the "best one", this commit adds code to scan the known NDK locations in search of the preferred version. Given an `AndroidSdkInfo` creation of: var info = new AndroidSdkInfo (logger:logger, androidSdkPath: sdkPath, androidNdkPath: ndkPath, javaSdkPath: javaPath); var usedNdkPath = info.AndroidNdkPath; If `ndkPath` is not `null` and otherwise valid, then `usedNdkPath` is `ndkPath`. If `ndkPath` is `null` or is otherwise invalid (missing `ndk-stack`, etc.), then we search for an `info.AndroidNdkPath` value as follows: 1. If `androidSdkPath` is not `null` and valid, then we check for Android SDK-relative NDK locations, in: * `{androidSdkPath}/ndk/*` * `{androidSdkPath}/ndk-bundle` For each found SDK-relative NDK directory, we filter out NDKs for which we cannot determine the package version, as well as those which are "too old" (< `MinimumCompatibleNDKMajorVersion`) or "too new" (> `MaximumCompatibleNDKMajorVersion`), currently r22. We prefer the NDK location with the highest version number. 2. If `androidSdkPath` is not `null` and valid and if there are no Android SDK-relative NDK locations, then we use the user-selected "preferred NDK location". See also `AndroidSdkInfo.SetPreferredAndroidNdkPath()`. 3. If `androidSdkPath` is not `null` and valid and if the preferred NDK location isn't set or is invalid, then we check directories specified in `$PATH`, and use the directory which contains `ndk-stack`. 4. If `androidSdkPath` is not `null` and valid and `$PATH` didn't contain `ndk-stack`, then we continue looking for NDK locations within the Android SDK locations specified by the `$ANDROID_HOME` and `$ANDROID_SDK_ROOT` environment variables. As with (1), these likewise look for e.g. `${ANDROID_HOME}/ndk/*` or `${ANDROID_SDK_ROOT}/ndk-bundle` directories and select the NDK with the highest supported version. 5. If `androidSdkPath` is `null`, then *first* we try to find a valid Android SDK directory, using on Unix: a. The preferred Android SDK directory; see also `AndroidSdkInfo.SetPreferredAndroidSdkPath(). b. The `$ANDROID_HOME` and `ANDROID_SDK_ROOT` environment variables. c. Directories within `$PATH` that contain `adb`. Once an Android SDK is found, steps (1)…(4) are performed. In (1) and (4), we now look for the Android SDK packages containing the NDK. There are two kinds of such packages: * `ndk-bundle` is the older package which allows for installation of only one NDK inside the SDK directory * `ndk/*` is a newer package which allows for installation of several NDK versions in parallel. Each subdirectory of `ndk` is an `X.Y.Z` version number of the NDK. In each of these directories we look for the `source.properties` file from which we then extract the NDK version and then we sort thus discovered NDK instances using their version as the key, in the descending order. The latest compatible (currently: less than 22 and more than 15) version is selected and its path returned to the caller.
jonpryor
pushed a commit
to dotnet/android-tools
that referenced
this issue
Jan 20, 2021
Context: actions/runner-images#2420 Context: dotnet/android#5499 Context: dotnet/android#5526 Context: android/ndk#1427 Context: https://developer.android.com/studio/command-line/variables#envar Xamarin.Android is not (yet) compatible with the recently released Android NDK r22 version. Azure build images have recently rolled out an update which includes NDK r22 and, thus, it breaks builds for customers using any form of Xamarin.Android AOT build. In an attempt to detect broken/incompatible NDK versions as well as to select the "best one", this commit adds code to scan the known NDK locations in search of the preferred version. Given an `AndroidSdkInfo` creation of: var info = new AndroidSdkInfo (logger:logger, androidSdkPath: sdkPath, androidNdkPath: ndkPath, javaSdkPath: javaPath); var usedNdkPath = info.AndroidNdkPath; If `ndkPath` is not `null` and otherwise valid, then `usedNdkPath` is `ndkPath`. If `ndkPath` is `null` or is otherwise invalid (missing `ndk-stack`, etc.), then we search for an `info.AndroidNdkPath` value as follows: 1. If `androidSdkPath` is not `null` and valid, then we check for Android SDK-relative NDK locations, in: * `{androidSdkPath}/ndk/*` * `{androidSdkPath}/ndk-bundle` For each found SDK-relative NDK directory, we filter out NDKs for which we cannot determine the package version, as well as those which are "too old" (< `MinimumCompatibleNDKMajorVersion`) or "too new" (> `MaximumCompatibleNDKMajorVersion`), currently r22. We prefer the NDK location with the highest version number. 2. If `androidSdkPath` is not `null` and valid and if there are no Android SDK-relative NDK locations, then we use the user-selected "preferred NDK location". See also `AndroidSdkInfo.SetPreferredAndroidNdkPath()`. 3. If `androidSdkPath` is not `null` and valid and if the preferred NDK location isn't set or is invalid, then we check directories specified in `$PATH`, and use the directory which contains `ndk-stack`. 4. If `androidSdkPath` is not `null` and valid and `$PATH` didn't contain `ndk-stack`, then we continue looking for NDK locations within the Android SDK locations specified by the `$ANDROID_HOME` and `$ANDROID_SDK_ROOT` environment variables. As with (1), these likewise look for e.g. `${ANDROID_HOME}/ndk/*` or `${ANDROID_SDK_ROOT}/ndk-bundle` directories and select the NDK with the highest supported version. 5. If `androidSdkPath` is `null`, then *first* we try to find a valid Android SDK directory, using on Unix: a. The preferred Android SDK directory; see also `AndroidSdkInfo.SetPreferredAndroidSdkPath(). b. The `$ANDROID_HOME` and `ANDROID_SDK_ROOT` environment variables. c. Directories within `$PATH` that contain `adb`. Once an Android SDK is found, steps (1)…(4) are performed. In (1) and (4), we now look for the Android SDK packages containing the NDK. There are two kinds of such packages: * `ndk-bundle` is the older package which allows for installation of only one NDK inside the SDK directory * `ndk/*` is a newer package which allows for installation of several NDK versions in parallel. Each subdirectory of `ndk` is an `X.Y.Z` version number of the NDK. In each of these directories we look for the `source.properties` file from which we then extract the NDK version and then we sort thus discovered NDK instances using their version as the key, in the descending order. The latest compatible (currently: less than 22 and more than 15) version is selected and its path returned to the caller.
Since you have the environment, can you help test what is the minimum cmake version that this issue is fixed? Seems we need to bump this a little bit. Or maybe we can conservatively make it 3.19? |
The minimun version is 3.19.0. I changed that value to 3.19 and it worked fine. |
Thanks seems what we need is cmake 5192 and cmake 5058? I'll update toolchain file. |
|
(Change merged in https://android-review.googlesource.com/c/platform/ndk/+/1557260) |
jonpryor
pushed a commit
to dotnet/android
that referenced
this issue
Feb 9, 2021
Context: mono/mono#20606 Context: https://github.com/android/ndk/wiki/Changelog-r22 Changes: mono/mono@5e9cb6d...c66141a * mono/mono@c66141a8c7b: [AOT] Make native linker name configurable (#20816) * mono/mono@64368a00d85: Bump msbuild * mono/mono@364c87b1a7a: Disable acceptance-tests/roslyn.mk * mono/mono@9b139294c04: Bump msbuild to fix build break * mono/mono@b9c050faf2e: [aot] Quote the -Wl,install_name argument to clang. (#20660) Add support for using [NDK r22][0] when building Xamarin.Android apps. The most important NDK changes: * GNU binutils are deprecated (but still used) * LLVM 11 is used for the toolchain * LLD is now the default linker Mono changes: * Allow the `ld` name to be configurable, as we now prefer e.g. `aarch64-linux-android-ld.gold`, not `aarch64-linux-android-ld`, as the latter is LLD and not binutils. Xamarin.Android CI changes: * All the binutils tools are listed in a single location, within `xaprepare`'s `Configurables.Defaults`. * Host NDK binutils are installed by `xaprepare` instead of by Xamarin.Android.Build.Tools * `UBSAN` checked builds now require `RTTI` and exceptions. Xamarin.Android App build changes: * The `<Aot/>` task now supports using NDK r22 TODO: * Add a nightly job (916d24b) which builds Xamarin.Android apps against NDK r21, to ensure we don't break support. * NDK r22's [`cmake` doesn't work on Windows][1]. We have worked around this by copying the fixed `android.toolchain.cmake` as `src-ThirdParty/ndk/android.toolchain.cmake.ndk_r21.3`. Once NDK r22 properly includes the fix, we should remove `src-ThirdParty/ndk/android.toolchain.cmake.ndk_r21.3`. [0]: https://github.com/android/ndk/wiki/Changelog-r22 [1]: android/ndk#1427
grendello
added a commit
to grendello/xamarin-android
that referenced
this issue
Mar 25, 2021
Context: https://github.com/android/ndk/wiki/Changelog-r22#r22b Context: android/ndk#1427 Upstream changes: * [Issue 1377][0]: Fix Clang backend crash in register scavenging. * [Issue 1388][1]: Fixed LLVM's incorrect conversion to list files for long command lines on Windows. * [Issue 1391][2]: Fixed missing symbols from libz.a. * [Issue 1427][3]: Fixed Clang auto-detection for CMake 3.19 and older for Windows. Fix for issue 1427 above allows us to remove the older Android cmake toolchain file we've been using to work around the bug on Windows platforms. [0]: android/ndk#1377 [1]: android/ndk#1388 [2]: android/ndk#1391 [3]: android/ndk#1427
jonpryor
pushed a commit
to dotnet/android
that referenced
this issue
Mar 26, 2021
Context: https://github.com/android/ndk/wiki/Changelog-r22#r22b Context: android/ndk#1427 Upstream changes: * [Issue 1377][0]: Fix Clang backend crash in register scavenging. * [Issue 1388][1]: Fixed LLVM's incorrect conversion to list files for long command lines on Windows. * [Issue 1391][2]: Fixed missing symbols from `libz.a`. * [Issue 1427][3]: Fixed Clang auto-detection for CMake 3.19 and older for Windows. The fix for issue 1427 above allows us to remove the older Android cmake toolchain file we've been using (accc846) to work around the bug on Windows platforms. [0]: android/ndk#1377 [1]: android/ndk#1388 [2]: android/ndk#1391 [3]: android/ndk#1427
|
This causes lots of warnings with every build now; cmake 3.10 is the latest the Android SDK/NDK offers. |
|
Did you update? As far as we know this is fixed. |
|
Update what? Update the NDK to 22.1? Yes, which is when the new build warnings begun to show up. Update cmake? No, because cmake 3.10 is the latest version that is possible to use with the SDK/NDK. Which, then, is the very cause of this warning because the NDK 22.1 really wants cmake 3.19 or newer. You should make it possible to comply with the raised requirements before you raise them ☺ |
|
The fix does not require CMake 3.19. The fix was specifically to make this work with versions of CMake earlier than 3.19. We do not have enough information to investigate further. We need a repro case. As for getting new versions of CMake into the SDK, you're talking to the wrong people. We only maintain the NDK. File a bug against Studio. |
|
https://issuetracker.google.com/177245495 seems to be the existing public bug. looks like they have 3.18 in the beta channel. |
|
Dan Albert dixit:
The fix does not require CMake 3.19.
Yes, I know. But you seem to not have read my initial comment
here, which says: the fix now causes tons of build warnings
in every single project.
(Incidentally even in whose where 22.0 worked fine, e.g. on GNU/Linux.)
As for getting new versions of CMake into the SDK, you're talking to
the wrong people. We only maintain the NDK.
Doesn’t mean it’s right to release a new version of the NDK that
effectively requires something that’s not available yet.
bye,
//mirabilos
--
Sometimes they [people] care too much: pretty printers [and syntax highligh-
ting, d.A.] mechanically produce pretty output that accentuates irrelevant
detail in the program, which is as sensible as putting all the prepositions
in English text in bold font. -- Rob Pike in "Notes on Programming in C"
|
Oh! You didn't say the fix caused this, you just said "this", which I took to mean "this bug". That makes sense now.
Nothing is being required here. It's a warning, not an error. That warning is there for a good reason. We used to have bugs filed often for things that happened as a result of compiler ID bypass. The warning is there to communicate "yeah, nothing the NDK can do about it, you need to use a newer version of CMake." The change in r22b was to raise the minimum version of CMake that was required for using the built-in compiler ID behavior, because prior to 3.19 there were bugs (that's the original report here). We can't revert this to fix the warnings without breaking builds. FWIW you don't need to wait for the SDK to get 3.20. You can use any version of CMake you want: https://developer.android.com/studio/projects/install-ndk#vanilla_cmake. |
|
Dan Albert dixit:
FWIW you don't need to wait for the SDK to get 3.20. You can use any
version of CMake you want:
https://developer.android.com/studio/projects/install-ndk#vanilla_cmake.
Ah, good to know. But I cannot require that of everyone who wants
to use the software… though, if it’s just warning…
|
grendello
added a commit
to grendello/runtime
that referenced
this issue
Apr 26, 2021
Context: android/ndk#1427 (comment) Android NDK r22 changed the tree layout by moving `sysroot` from the NDK root directory to a subdirectory in the toolchains tree and also by removing per-platform library/header directories from their previous location to one under the sysroot above. MonoVM was built using the CMake's built-in Android NDK support which, alas, is not compatible with NDK r22 before CMake version 3.19. It is therefore better to use the Android CMake toolchain definition script shipped with the NDK, so that the build works regardless of CMake version installed on the system. The r22 toolchain will complain about older CMake versions: An old version of CMake is being used that cannot automatically detect compiler attributes. Compiler identification is being bypassed. Some values may be wrong or missing. Update to CMake 3.19 or newer to use CMake's built-in compiler identification. This warning can be safely ignored.
marek-safar
pushed a commit
to dotnet/runtime
that referenced
this issue
Apr 28, 2021
Context: android/ndk#1427 (comment) Android NDK r22 changed the tree layout by moving `sysroot` from the NDK root directory to a subdirectory in the toolchains tree and also by removing per-platform library/header directories from their previous location to one under the sysroot above. MonoVM was built using the CMake's built-in Android NDK support which, alas, is not compatible with NDK r22 before CMake version 3.19. It is therefore better to use the Android CMake toolchain definition script shipped with the NDK, so that the build works regardless of CMake version installed on the system. The r22 toolchain will complain about older CMake versions: An old version of CMake is being used that cannot automatically detect compiler attributes. Compiler identification is being bypassed. Some values may be wrong or missing. Update to CMake 3.19 or newer to use CMake's built-in compiler identification. This warning can be safely ignored.
|
Dan Albert dixit:
FWIW you don't need to wait for the SDK to get 3.20. You can use any
version of CMake you want:
https://developer.android.com/studio/projects/install-ndk#vanilla_cmake.
Heh. Debian also only has 3.18.4 so there is no newer version available
in a way acceptable to machine usage policy at work…
bye,
//mirabilos
--
Infrastrukturexperte • tarent solutions GmbH
Am Dickobskreuz 10, D-53121 Bonn • http://www.tarent.de/
Telephon +49 228 54881-393 • Fax: +49 228 54881-235
HRB AG Bonn 5168 • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
Xamarin.Android has been using Android SDK's cmake for quite a while now and
until NDK r22 everything has worked as expected. With r22 the Unix (Linux and macOS) builds still work fine, however the Windows build breaks because
cmakeis not able to properly detect the C/C++ compilers (please see the attached error and output logs). Not only that, it appears that theWIN32cmakevariable is not set at all, insteadMINGWis set.We normally use
cmakefrom the SDK (version 3.10.2), but I also tried withcmake3.18 shipped with VS2019, to the same effect. OurCMakeLists.txt, PR to switch XA to NDK r22, and a sample invocation line:The same
CMakeList.txt(and the samecmakeversion) works fine with r21 on WindowsEnvironment Details
Not all of these will be relevant to every bug, but please provide as much
information as you can.
CMakeError.log.txt
CMakeOutput.log.txt
The text was updated successfully, but these errors were encountered: