-
Notifications
You must be signed in to change notification settings - Fork 533
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Xamarin.Android.Build.Tasks] Implement a new process for defining versionCode. #692
Conversation
{ | ||
var regex = new Regex ("\\{(?<key>([A-Za-z]*)):?0*[\\}]"); | ||
var kvp = new Dictionary<string, int> (); | ||
foreach (var item in versionCodeProperties?.Split (new char [] { ';' }) ?? new string [0]) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should split on both ;
and :
, or just :
.
;
can't be specified on the command-line as part of a property value, as it's the property separator:
msbuild /p:Foo=Bar;Baz=Qux
|
||
public void CalculateVersionCode (string currentAbi, string versionCodePattern, string versionCodeProperties) | ||
{ | ||
var regex = new Regex ("\\{(?<key>([A-Za-z]*)):?0*[\\}]"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really want (?<key>([A-Za-z]*))
? That would match no characters, allowing a match of {}
. Is that sensible?
Additionally, does the 0*
make sense? That looks like it should be any format string, so what would be wrong with e.g. {versionCode:x8}
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After thinking a bit more I think we should only support integer format padding strings.. Since the version code MUST be an integer it makes no sense to support anything else.
So we should only support
{0:00000}
{0:D4}
for padding.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The problem is that there are multiple format strings which result in an integer, e.g. g
, and I don't think that whitelisting is sane.
I'm also not sure that it matters, necessarily? If they specify e.g. x4
and get a letter in the resulting versionCode, then aapt
/etc. will complain later about the invalid version. So long as the resulting error message is sensible, it shouldn't matter where it errors, so long as it does.
Documentation/build_process.md
Outdated
Some examples, if `abi` is `armeabi` and `versionCode` in the manifest | ||
is `123` | ||
|
||
`{abi}{versionCode}` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This line shouldn't begin/end with ````` (backtick), unless you want the property to contain backticks...
msbuild '/p:AndroidVersionCodePattern=`{abi}{versionCode}`'
I assume you don't, so this line and the line ~10 lines down likewise shouldn't begin/end with backticks. These lines are already indented extra, which makes them formatted as code.
Documentation/build_process.md
Outdated
- **versionCode** &ndsh; Uses the version code direrctly from | ||
`Properties\AndroidManifest.xml`. | ||
|
||
You can define custom items using the [AndroidVersionCodeProperties] (#AndroidVersionCodeProperties) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There should be no space between the ]
and the (
.
Documentation/build_process.md
Outdated
|
||
Pre defined key items | ||
|
||
- **abi** &ndsh; Inserts the targetted abi for the app |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be –
, not &ndsh;
; note the missing a
.
Documentation/build_process.md
Outdated
- **AndroidVersionCodePattern** &ndsh; A string property which allows | ||
the developer to customize the `versionCode` in the manifest when splitting | ||
up the apk by abi. | ||
See [Creating_the_Version_Code_for_the_APK](https://developer.xamarin.com/guides/android/advanced_topics/build-abi-specific-apks/#Creating_the_Version_Code_for_the_APK) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This shouldn't contain _
s:
See [Creating the Version Code for the APK](https://developer.xamarin.com/guides/android/advanced_topics/build-abi-specific-apks/#Creating_the_Version_Code_for_the_APK)
Documentation/build_process.md
Outdated
|
||
Added in Xamarin.Android 7.2. | ||
- **AndroidVersionCodeProperties** &ndsh; A string property which allows | ||
the developer to define custom items to use with the [AndroidVersionCodePattern] (#AndroidVersionCodePattern). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto here; no space between ]
and (
.
Documentation/build_process.md
Outdated
property. | ||
|
||
Added in Xamarin.Android 7.2. | ||
- **AndroidVersionCodeProperties** &ndsh; A string property which allows |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is documented, but I don't see any tests which exercise this property.
I think we should restrict it only to "0" and "Dx" as they are the ones
that make sense (and document that).
…On 13 July 2017 at 14:47, Jonathan Pryor ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In src/Xamarin.Android.Build.Tasks/Utilities/ManifestDocument.cs
<#692 (comment)>
:
> }
code |= GetAbiCode (abi) << 16;
VersionCode = code.ToString ();
}
+
+ public void CalculateVersionCode (string currentAbi, string versionCodePattern, string versionCodeProperties)
+ {
+ var regex = new Regex ("\\{(?<key>([A-Za-z]*)):?0*[\\}]");
The problem is that there are multiple format strings which result in an
integer, e.g. g, and I don't think that whitelisting is sane.
I'm also not sure that it matters, necessarily? If they specify e.g. x4
and get a letter in the resulting versionCode, then aapt/etc. will
complain later about the invalid version. So long as the resulting error
message is sensible, it shouldn't matter *where* it errors, so long as it
does.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#692 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAxeefzM4TshWx7zaDXOEnmZK-CKXFKLks5sNh_ygaJpZM4OV5Cm>
.
|
b07d47f
to
f1be8a7
Compare
public string MinimumSdk { | ||
get { | ||
var uses = doc.Root.Element ("uses-sdk"); | ||
if (uses?.Attribute (androidNs + "minSdkVersion") == null) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given that you actually want the minSdkVersion
attribute value, why not look for that instead of the element?
var minAttr = doc.Root.Element ("uses-sdk")?.Attribute (androidNs + "minSdkVersion");
if (minAttr == null) {
...
}
return minAttr.Value;
@@ -68,6 +70,19 @@ internal class ManifestDocument | |||
doc.Root.SetAttributeValue (androidNs + "versionCode", value); | |||
} | |||
} | |||
public string MinimumSdk { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps this should be a GetMinimumSdk()
method, not a property, given that this property isn't just a field access? It would inform callers that they may want to cache the value instead of calling the property repeatedly.
Documentation/build_process.md
Outdated
|
||
{abi}{versionCode} | ||
|
||
will prodice a versionCode of `1123`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be:
...will produce [sic] a versionCode of
11123
when$(AndroidCreatePackagePerAbi)
is True, otherwise will produce a value of123
.
Documentation/build_process.md
Outdated
|
||
will prodice a versionCode of `1123`. | ||
If `abi` is `x86_64` and `versionCode` in the manifest | ||
is `44`. This will produce `544`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This similarly should mention $(AndroidCreatePackagePerAbi)
.
Documentation/build_process.md
Outdated
Added in Xamarin.Android 7.2. | ||
|
||
- **AndroidVersionCodeProperties** – A string property which allows | ||
the developer to define custom items to use with the [AndroidVersionCodePattern](#AndroidVersionCodePattern). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like a prior documentation
repro which moved this to be a list instead of a table broke the hyperlinks, or, hyperlinks now require post-processing/hosting on developer.xamarin.com
Specifically, this link doesn't go anywhere when simply rendered as HTML.
Perhaps this should add <a name="..." />
so that these links work?
…rsionCode. Context https://bugzilla.xamarin.com/show_bug.cgi?id=51620 Context https://bugzilla.xamarin.com/show_bug.cgi?id=51618 Context https://bugzilla.xamarin.com/show_bug.cgi?id=51145 Our current version system for multiple apk's for each Abi is a bit broken [1]. If a user for example has a versionCode set which is 123 the final version code for an x86_64 build ends up as 327803. This is completely transparent to the user and also does not follow the guidance in the documentation at [1] and [2]. So we need a new system :) but as usual we have to support the old system. So we are introducing a new system which is more flexible. This will only apply when the `$(AndroidCreatePackagePerAbi)` is set to `True`. The new system has two new properties <AndroidVersionCodePattern/> <AndroidVersionCodeProperties/> The first allows the developer to define the Pattern to be used for the versonCode. The pattern will be made up of a format string which will contain keys. These keys will be replaced with values form one of the known keys or a custom user defined one. We define a few known key values - abi : The current target abi converted to an int where - 'armeabi' = 1, - 'armeabi-v7a' = 2, - 'x86' = 3, - 'arm64-v8a' = 4, - 'x86_64' = 5, - minSDK : The minSDK value from the manifest or 11 if not present. - versionCode : The versionCode from the manifest. With these keys the user can define a pattern of {abi}{minSDK}{versionCode} or if they way to include zero padding they can use {abi}{minSDK}{versionCode:D4} similar to the left padding formats used in string.Format (). Users can also use the `$(AndroidVersionCodeProperties)` property to define new custom keys. This string will be in the form of a semi-colon delimited key=value pairs. For example foo=12;bar=$(SomeBuildProperty) when can then be used in the pattern. {abi}{foo}{bar}{versionCode} Lets work through an example. The user defines a version code of '123' in the manifest and enables `$(AndroidCreatePackagePerAbi)`. They define a `$(AndroidVersionCodePattern)` of `{abi}{versionCode:D5}`. This will result in the following version code being produced for the 'x86' build. 300123 The first 3 is the `{abi}` value. The rest is the left zero padded versionCode. A slightly more complex pattern would be `{abi}{minSDK:D2}{versionCode:D4}` which would produce 3140123 if the minimumSdk value was set to API 14. A more real life example mgiht be as follows. A user wants to use the `Build` value from the AssemblyInfo.cs . They define the following target ```xml <Target Name="_GetBuild" AfterTargets="Compile"> <GetAssemblyIdentity AssemblyFiles="Foo.dll"> <Output TaskParameter="Assemblies" ItemName="MasterVersion"/> </GetAssemblyIdentity> <PropertyGroup> <BuildVersion>$([System.Version]::Parse(%(MasterVersion.Version)).Build)</BuildVersion> </PropertyGroup> </Target> ``` This extracts the build version from the built assembly. They can then define a pattern of {abi}{minSDK}{build:D4} and set the properties to build=$(BuildVersion) Given similar properties from the previous example e.g abi=x86 and minSDk=14, this will result in the follwing output (assuming the `Build` value was 3421). 3143421 [1] https://developer.xamarin.com/guides/android/advanced_topics/build-abi-specific-apks/ [2] https://developer.android.com/google/play/publishing/multiple-apks.html#Rules
…rsionCode. (#692) Context: https://bugzilla.xamarin.com/show_bug.cgi?id=51620 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=51618 Context: https://bugzilla.xamarin.com/show_bug.cgi?id=51145 Our current version system for multiple apk's for each Abi is a bit broken [1]. If a user for example has a versionCode set which is 123 the final version code for an x86_64 build ends up as 327803. This is completely transparent to the user and also does not follow the guidance in the documentation at [1] and [2]. So we need a new system :) but as usual we have to support the old system. So we are introducing a new system which is more flexible. This will only apply when the `$(AndroidCreatePackagePerAbi)` is set to `True`. The new system has two new properties <AndroidVersionCodePattern/> <AndroidVersionCodeProperties/> The first allows the developer to define the Pattern to be used for the versonCode. The pattern will be made up of a format string which will contain keys. These keys will be replaced with values form one of the known keys or a custom user defined one. We define a few known key values - abi : The current target abi converted to an int where - 'armeabi' = 1, - 'armeabi-v7a' = 2, - 'x86' = 3, - 'arm64-v8a' = 4, - 'x86_64' = 5, - minSDK : The minSDK value from the manifest or 11 if not present. - versionCode : The versionCode from the manifest. With these keys the user can define a pattern of {abi}{minSDK}{versionCode} or if they way to include zero padding they can use {abi}{minSDK}{versionCode:D4} similar to the left padding formats used in string.Format (). Users can also use the `$(AndroidVersionCodeProperties)` property to define new custom keys. This string will be in the form of a semi-colon delimited key=value pairs. For example foo=12;bar=$(SomeBuildProperty) when can then be used in the pattern. {abi}{foo}{bar}{versionCode} Lets work through an example. The user defines a version code of '123' in the manifest and enables `$(AndroidCreatePackagePerAbi)`. They define a `$(AndroidVersionCodePattern)` of `{abi}{versionCode:D5}`. This will result in the following version code being produced for the 'x86' build. 300123 The first 3 is the `{abi}` value. The rest is the left zero padded versionCode. A slightly more complex pattern would be `{abi}{minSDK:D2}{versionCode:D4}` which would produce 3140123 if the minimumSdk value was set to API 14. A more real life example mgiht be as follows. A user wants to use the `Build` value from the AssemblyInfo.cs . They define the following target ```xml <Target Name="_GetBuild" AfterTargets="Compile"> <GetAssemblyIdentity AssemblyFiles="Foo.dll"> <Output TaskParameter="Assemblies" ItemName="MasterVersion"/> </GetAssemblyIdentity> <PropertyGroup> <BuildVersion>$([System.Version]::Parse(%(MasterVersion.Version)).Build)</BuildVersion> </PropertyGroup> </Target> ``` This extracts the build version from the built assembly. They can then define a pattern of {abi}{minSDK}{build:D4} and set the properties to build=$(BuildVersion) Given similar properties from the previous example e.g abi=x86 and minSDk=14, this will result in the follwing output (assuming the `Build` value was 3421). 3143421 [1] https://developer.xamarin.com/guides/android/advanced_topics/build-abi-specific-apks/ [2] https://developer.android.com/google/play/publishing/multiple-apks.html#Rules
Fixes: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1139578 Context: dotnet/java-interop#691 Context: https://liquid.microsoft.com/Web/Object/Read/ms.security/Requirements/Microsoft.Security.SystemsADM.10039#guide Changes: dotnet/java-interop@007b35b...08ff4db * dotnet/java-interop@08ff4db1: [java-interop, Java.Interop] Securely load native libs (dotnet#691) * dotnet/java-interop@09c68911: [generator] Clean up generated whitespace (dotnet#692) * dotnet/java-interop@d664e90e: [generator] Make warning and error messages localizable. (dotnet#689) For improved security, the guidance is that when loading `.dll` files on Windows, the default [`LoadLibrary()`][0] behavior is to be *avoided*, as the [Dynamic-Link Library Search Order][1] includes the "current working directory" and directories listed in `%PATH%`, both of which are under control of an "attacker" and not the application. Instead, [`LoadLibraryEx()`][2] should be used, providing a set of `LOAD_LIBRARY_SEARCH_*` values which constrain the set of directories that will be searched for the library (if a full path is not used) and/or the library's dependencies. Historically, Xamarin.Android hasn't directly called any `LoadLibrary()` function. Instead, xamarin-android called [**dlopen**(3)][3], then used dlfcn-win32/dlfcn-win32@ef7e412d to implement `dlopen()` in terms of `LoadLibraryEx()`. Unfortunately, dlfcn-win32/dlfcn-win32@ef7e412d calls `LoadLibraryEx()` with `LOAD_WITH_ALTERED_SEARCH_PATH`, which while a security improvement, does not go far enough for our internal requirements. Migrate away from dlfcn-win32 and instead use the new `java-interop-dlfcn.h` family of functions added in dotnet/java-interop@08ff4db1. These new functions appropriately use `LoadLibraryEx()` with a constrained set of `LOAD_LIBRARY_SEARCH_*` flags. *Non-obvious semantic note*: On Mono for all platforms, P/Invoking `__Internal` is equivalent to trying to load the path `NULL`, e.g. on macOS `dlopen(NULL, …)`. However, only Android allows a custom "`loader_func`" to intercept the attempted load of `NULL` and "do something" with it, which xamarin-android *does* intercept, "remapping" `NULL` to `libxa-internal-api.so`. On all other platforms, `loader_func` cannot intercept this invocation. …which made for an "interesting" interaction: on macOS, if we used: api_dso_handle = java_interop_lib_load (dso_path.get (), JAVA_INTEROP_LIB_LOAD_LOCALLY, nullptr); then the macOS Designer integration tests would fail! Renderer >> 4 [monodroid] Calling into managed runtime init Renderer (error) >> Renderer (error) >> Unhandled Exception: Renderer (error) >> System.EntryPointNotFoundException: java_interop_jnienv_get_java_vm assembly:<unknown assembly> type:<unknown type> member:(null) Renderer (error) >> at (wrapper managed-to-native) Java.Interop.NativeMethods.java_interop_jnienv_get_java_vm(intptr,intptr&) Renderer (error) >> at Java.Interop.JniEnvironment+References.GetJavaVM (System.IntPtr jnienv, System.IntPtr& vm) [0x00000] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Java.Interop.JniEnvironmentInfo.set_EnvironmentPointer (System.IntPtr value) [0x00037] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Java.Interop.JniEnvironmentInfo..ctor (System.IntPtr environmentPointer, Java.Interop.JniRuntime runtime) [0x00006] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Java.Interop.JniRuntime..ctor (Java.Interop.JniRuntime+CreationOptions options) [0x0017b] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Android.Runtime.AndroidRuntime..ctor (System.IntPtr jnienv, System.IntPtr vm, System.Boolean allocNewObjectSupported, System.IntPtr classLoader, System.IntPtr classLoader_loadClass, System.Boolean jniAddNativeMethodRegistrationAttributePresent) [0x00000] in /Users/builder/azdo/_work/4/s/xamarin-android/src/Mono.Android/Android.Runtime/AndroidRuntime.cs:25 In order for the macOS Designer integration tests to succeed, `libxa-internal-api.dylib` *must* be loaded as `RTLD_GLOBAL`, which is done via `JAVA_INTEROP_LIB_LOAD_GLOBALLY`. [0]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibrarya [1]: https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order [2]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexa [3]: https://linux.die.net/man/3/dlopen
Fixes: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1139578 Context: dotnet/java-interop#691 Context: https://liquid.microsoft.com/Web/Object/Read/ms.security/Requirements/Microsoft.Security.SystemsADM.10039#guide Changes: dotnet/java-interop@007b35b...08ff4db * dotnet/java-interop@08ff4db1: [java-interop, Java.Interop] Securely load native libs (dotnet#691) * dotnet/java-interop@09c68911: [generator] Clean up generated whitespace (dotnet#692) * dotnet/java-interop@d664e90e: [generator] Make warning and error messages localizable. (dotnet#689) For improved security, the guidance is that when loading `.dll` files on Windows, the default [`LoadLibrary()`][0] behavior is to be *avoided*, as the [Dynamic-Link Library Search Order][1] includes the "current working directory" and directories listed in `%PATH%`, both of which are under control of an "attacker" and not the application. Instead, [`LoadLibraryEx()`][2] should be used, providing a set of `LOAD_LIBRARY_SEARCH_*` values which constrain the set of directories that will be searched for the library (if a full path is not used) and/or the library's dependencies. Historically, Xamarin.Android hasn't directly called any `LoadLibrary()` function. Instead, xamarin-android called [**dlopen**(3)][3], then used dlfcn-win32/dlfcn-win32@ef7e412d to implement `dlopen()` in terms of `LoadLibraryEx()`. Unfortunately, dlfcn-win32/dlfcn-win32@ef7e412d calls `LoadLibraryEx()` with `LOAD_WITH_ALTERED_SEARCH_PATH`, which while a security improvement, does not go far enough for our internal requirements. Migrate away from dlfcn-win32 and instead use the new `java-interop-dlfcn.h` family of functions added in dotnet/java-interop@08ff4db1. These new functions appropriately use `LoadLibraryEx()` with a constrained set of `LOAD_LIBRARY_SEARCH_*` flags. *Non-obvious semantic note*: On Mono for all platforms, P/Invoking `__Internal` is equivalent to trying to load the path `NULL`, e.g. on macOS `dlopen(NULL, …)`. However, only Android allows a custom "`loader_func`" to intercept the attempted load of `NULL` and "do something" with it, which xamarin-android *does* intercept, "remapping" `NULL` to `libxa-internal-api.so`. On all other platforms, `loader_func` cannot intercept this invocation. …which made for an "interesting" interaction: on macOS, if we used: api_dso_handle = java_interop_lib_load (dso_path.get (), JAVA_INTEROP_LIB_LOAD_LOCALLY, nullptr); then the macOS Designer integration tests would fail! Renderer >> 4 [monodroid] Calling into managed runtime init Renderer (error) >> Renderer (error) >> Unhandled Exception: Renderer (error) >> System.EntryPointNotFoundException: java_interop_jnienv_get_java_vm assembly:<unknown assembly> type:<unknown type> member:(null) Renderer (error) >> at (wrapper managed-to-native) Java.Interop.NativeMethods.java_interop_jnienv_get_java_vm(intptr,intptr&) Renderer (error) >> at Java.Interop.JniEnvironment+References.GetJavaVM (System.IntPtr jnienv, System.IntPtr& vm) [0x00000] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Java.Interop.JniEnvironmentInfo.set_EnvironmentPointer (System.IntPtr value) [0x00037] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Java.Interop.JniEnvironmentInfo..ctor (System.IntPtr environmentPointer, Java.Interop.JniRuntime runtime) [0x00006] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Java.Interop.JniRuntime..ctor (Java.Interop.JniRuntime+CreationOptions options) [0x0017b] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Android.Runtime.AndroidRuntime..ctor (System.IntPtr jnienv, System.IntPtr vm, System.Boolean allocNewObjectSupported, System.IntPtr classLoader, System.IntPtr classLoader_loadClass, System.Boolean jniAddNativeMethodRegistrationAttributePresent) [0x00000] in /Users/builder/azdo/_work/4/s/xamarin-android/src/Mono.Android/Android.Runtime/AndroidRuntime.cs:25 In order for the macOS Designer integration tests to succeed, `libxa-internal-api.dylib` *must* be loaded as `RTLD_GLOBAL`, which is done via `JAVA_INTEROP_LIB_LOAD_GLOBALLY`. [0]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibrarya [1]: https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order [2]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexa [3]: https://linux.die.net/man/3/dlopen
Fixes: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1139578 Context: dotnet/java-interop#691 Context: https://liquid.microsoft.com/Web/Object/Read/ms.security/Requirements/Microsoft.Security.SystemsADM.10039#guide Changes: dotnet/java-interop@007b35b...dcaf794 * dotnet/java-interop@dcaf794d: [generator] Fix error and warning codes values (dotnet#697) * dotnet/java-interop@08ff4db1: [java-interop, Java.Interop] Securely load native libs (dotnet#691) * dotnet/java-interop@09c68911: [generator] Clean up generated whitespace (dotnet#692) * dotnet/java-interop@d664e90e: [generator] Make warning and error messages localizable. (dotnet#689) For improved security, the guidance is that when loading `.dll` files on Windows, the default [`LoadLibrary()`][0] behavior is to be *avoided*, as the [Dynamic-Link Library Search Order][1] includes the "current working directory" and directories listed in `%PATH%`, both of which are under control of an "attacker" and not the application. Instead, [`LoadLibraryEx()`][2] should be used, providing a set of `LOAD_LIBRARY_SEARCH_*` values which constrain the set of directories that will be searched for the library (if a full path is not used) and/or the library's dependencies. Historically, Xamarin.Android hasn't directly called any `LoadLibrary()` function. Instead, xamarin-android called [**dlopen**(3)][3], then used dlfcn-win32/dlfcn-win32@ef7e412d to implement `dlopen()` in terms of `LoadLibraryEx()`. Unfortunately, dlfcn-win32/dlfcn-win32@ef7e412d calls `LoadLibraryEx()` with `LOAD_WITH_ALTERED_SEARCH_PATH`, which while a security improvement, does not go far enough for our internal requirements. Migrate away from dlfcn-win32 and instead use the new `java-interop-dlfcn.h` family of functions added in dotnet/java-interop@08ff4db1. These new functions appropriately use `LoadLibraryEx()` with a constrained set of `LOAD_LIBRARY_SEARCH_*` flags. *Non-obvious semantic note*: On Mono for all platforms, P/Invoking `__Internal` is equivalent to trying to load the path `NULL`, e.g. on macOS `dlopen(NULL, …)`. However, only Android allows a custom "`loader_func`" to intercept the attempted load of `NULL` and "do something" with it, which xamarin-android *does* intercept, "remapping" `NULL` to `libxa-internal-api.so`. On all other platforms, `loader_func` cannot intercept this invocation. …which made for an "interesting" interaction: on macOS, if we used: api_dso_handle = java_interop_lib_load (dso_path.get (), JAVA_INTEROP_LIB_LOAD_LOCALLY, nullptr); then the macOS Designer integration tests would fail! Renderer >> 4 [monodroid] Calling into managed runtime init Renderer (error) >> Renderer (error) >> Unhandled Exception: Renderer (error) >> System.EntryPointNotFoundException: java_interop_jnienv_get_java_vm assembly:<unknown assembly> type:<unknown type> member:(null) Renderer (error) >> at (wrapper managed-to-native) Java.Interop.NativeMethods.java_interop_jnienv_get_java_vm(intptr,intptr&) Renderer (error) >> at Java.Interop.JniEnvironment+References.GetJavaVM (System.IntPtr jnienv, System.IntPtr& vm) [0x00000] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Java.Interop.JniEnvironmentInfo.set_EnvironmentPointer (System.IntPtr value) [0x00037] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Java.Interop.JniEnvironmentInfo..ctor (System.IntPtr environmentPointer, Java.Interop.JniRuntime runtime) [0x00006] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Java.Interop.JniRuntime..ctor (Java.Interop.JniRuntime+CreationOptions options) [0x0017b] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Android.Runtime.AndroidRuntime..ctor (System.IntPtr jnienv, System.IntPtr vm, System.Boolean allocNewObjectSupported, System.IntPtr classLoader, System.IntPtr classLoader_loadClass, System.Boolean jniAddNativeMethodRegistrationAttributePresent) [0x00000] in /Users/builder/azdo/_work/4/s/xamarin-android/src/Mono.Android/Android.Runtime/AndroidRuntime.cs:25 In order for the macOS Designer integration tests to succeed, `libxa-internal-api.dylib` *must* be loaded as `RTLD_GLOBAL`, which is done via `JAVA_INTEROP_LIB_LOAD_GLOBALLY`. [0]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibrarya [1]: https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order [2]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexa [3]: https://linux.die.net/man/3/dlopen
Fixes: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1139578 Context: dotnet/java-interop#691 Context: https://liquid.microsoft.com/Web/Object/Read/ms.security/Requirements/Microsoft.Security.SystemsADM.10039#guide Changes: dotnet/java-interop@007b35b...dcaf794 * dotnet/java-interop@dcaf794d: [generator] Fix error and warning codes values (#697) * dotnet/java-interop@08ff4db1: [java-interop, Java.Interop] Securely load native libs (#691) * dotnet/java-interop@09c68911: [generator] Clean up generated whitespace (#692) * dotnet/java-interop@d664e90e: [generator] Make warning and error messages localizable. (#689) For improved security, the guidance is that when loading `.dll` files on Windows, the default [`LoadLibrary()`][0] behavior is to be *avoided*, as the [Dynamic-Link Library Search Order][1] includes the "current working directory" and directories listed in `%PATH%`, both of which are under control of an "attacker" and not the application. Instead, [`LoadLibraryEx()`][2] should be used, providing a set of `LOAD_LIBRARY_SEARCH_*` values which constrain the set of directories that will be searched for the library (if a full path is not used) and/or the library's dependencies. Historically, Xamarin.Android hasn't directly called any `LoadLibrary()` function. Instead, xamarin-android called [**dlopen**(3)][3], then used dlfcn-win32/dlfcn-win32@ef7e412d to implement `dlopen()` in terms of `LoadLibraryEx()`. Unfortunately, dlfcn-win32/dlfcn-win32@ef7e412d calls `LoadLibraryEx()` with `LOAD_WITH_ALTERED_SEARCH_PATH`, which while a security improvement, does not go far enough for our internal requirements. Migrate away from dlfcn-win32 and instead use the new `java-interop-dlfcn.h` family of functions added in dotnet/java-interop@08ff4db1. These new functions appropriately use `LoadLibraryEx()` with a constrained set of `LOAD_LIBRARY_SEARCH_*` flags. *Non-obvious semantic note*: On Mono for all platforms, P/Invoking `__Internal` is equivalent to trying to load the path `NULL`, e.g. on macOS `dlopen(NULL, …)`. However, only Android allows a custom "`loader_func`" to intercept the attempted load of `NULL` and "do something" with it, which xamarin-android *does* intercept, "remapping" `NULL` to `libxa-internal-api.so`. On all other platforms, `loader_func` cannot intercept this invocation. …which made for an "interesting" interaction: on macOS, if we used: api_dso_handle = java_interop_lib_load (dso_path.get (), JAVA_INTEROP_LIB_LOAD_LOCALLY, nullptr); then the macOS Designer integration tests would fail! Renderer >> 4 [monodroid] Calling into managed runtime init Renderer (error) >> Renderer (error) >> Unhandled Exception: Renderer (error) >> System.EntryPointNotFoundException: java_interop_jnienv_get_java_vm assembly:<unknown assembly> type:<unknown type> member:(null) Renderer (error) >> at (wrapper managed-to-native) Java.Interop.NativeMethods.java_interop_jnienv_get_java_vm(intptr,intptr&) Renderer (error) >> at Java.Interop.JniEnvironment+References.GetJavaVM (System.IntPtr jnienv, System.IntPtr& vm) [0x00000] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Java.Interop.JniEnvironmentInfo.set_EnvironmentPointer (System.IntPtr value) [0x00037] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Java.Interop.JniEnvironmentInfo..ctor (System.IntPtr environmentPointer, Java.Interop.JniRuntime runtime) [0x00006] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Java.Interop.JniRuntime..ctor (Java.Interop.JniRuntime+CreationOptions options) [0x0017b] in <0f003a4904fd44d0a8cc6a63962ab40b>:0 Renderer (error) >> at Android.Runtime.AndroidRuntime..ctor (System.IntPtr jnienv, System.IntPtr vm, System.Boolean allocNewObjectSupported, System.IntPtr classLoader, System.IntPtr classLoader_loadClass, System.Boolean jniAddNativeMethodRegistrationAttributePresent) [0x00000] in /Users/builder/azdo/_work/4/s/xamarin-android/src/Mono.Android/Android.Runtime/AndroidRuntime.cs:25 In order for the macOS Designer integration tests to succeed, `libxa-internal-api.dylib` *must* be loaded as `RTLD_GLOBAL`, which is done via `JAVA_INTEROP_LIB_LOAD_GLOBALLY`. [0]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibrarya [1]: https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order [2]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexa [3]: https://linux.die.net/man/3/dlopen
Context https://bugzilla.xamarin.com/show_bug.cgi?id=51620
Context https://bugzilla.xamarin.com/show_bug.cgi?id=51618
Context https://bugzilla.xamarin.com/show_bug.cgi?id=51145
Our current version system for multiple apk's for each Abi
is a bit broken [1]. If a user for example has a versionCode
set which is 123 the final version code for an x86_64 build
ends up as 327803.
This is completely transparent to the user and also does not
follow the guidance in the documentation at [1] and [2].
So we need a new system :) but as usual we have to support the
old system. So we are introducing a new system which is more
flexible. This will only apply when the
$(AndroidCreatePackagePerAbi)
is set to
True
.The new system has two new properties
The first allows the developer to define the Pattern to be used
for the versonCode. The pattern will be made up of a format string
which will contain keys. These keys will be replaced with values
form one of the known keys or a custom user defined one.
We define a few known key values
abi : The current target abi converted to an int where
minSDK : The minSDK value from the manifest or 11 if not present.
versionCode : The versionCode from the manifest.
With these keys the user can define a pattern of
or if they way to include zero padding they can use
similar to the left padding formats used in string.Format ().
Users can also use the
$(AndroidVersionCodeProperties)
propertyto define new custom keys. This string will be in the form of a
semi-colon delimited key=value pairs. For example
when can then be used in the pattern.
Lets work through an example. The user defines a version code of '123'
in the manifest and enables
$(AndroidCreatePackagePerAbi)
. They definea
$(AndroidVersionCodePattern)
of{abi}{versionCode:D5}
.This will result in the following version code being produced for the
'x86' build.
The first 3 is the
{abi}
value. The rest is the left zero paddedversionCode.
A slightly more complex pattern would be
{abi}{minSDK:D2}{versionCode:D4}
which would produce
if the minimumSdk value was set to API 14.
A more real life example mgiht be as follows. A user wants to use the
Build
valuefrom the AssemblyInfo.cs . They define the following target
This extracts the build version from the built assembly. They can then define a
pattern of
and set the properties to
Given similar properties from the previous example e.g abi=x86 and minSDk=14,
this will result in the follwing output (assuming the
Build
value was 3421).[1] https://developer.xamarin.com/guides/android/advanced_topics/build-abi-specific-apks/
[2] https://developer.android.com/google/play/publishing/multiple-apks.html#Rules