Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Xamarin.Android.Build.Tasks] Implement a new process for defining versionCode. #692

Merged
merged 1 commit into from
Jul 27, 2017

Conversation

dellis1972
Copy link
Contributor

@dellis1972 dellis1972 commented Jul 12, 2017

Context https://bugzilla.xamarin.com/show_bug.cgi?id=51620
Context https://bugzilla.xamarin.com/show_bug.cgi?id=51618
Context https://bugzilla.xamarin.com/show_bug.cgi?id=51145

Our current version system for multiple apk's for each Abi
is a bit broken [1]. If a user for example has a versionCode
set which is 123 the final version code for an x86_64 build
ends up as 327803.
This is completely transparent to the user and also does not
follow the guidance in the documentation at [1] and [2].

So we need a new system :) but as usual we have to support the
old system. So we are introducing a new system which is more
flexible. This will only apply when the $(AndroidCreatePackagePerAbi)
is set to True.

The new system has two new properties

<AndroidVersionCodePattern/>
<AndroidVersionCodeProperties/>

The first allows the developer to define the Pattern to be used
for the versonCode. The pattern will be made up of a format string
which will contain keys. These keys will be replaced with values
form one of the known keys or a custom user defined one.

We define a few known key values

  • abi : The current target abi converted to an int where

    • 'armeabi' = 1,
    • 'armeabi-v7a' = 2,
    • 'x86' = 3,
    • 'arm64-v8a' = 4,
    • 'x86_64' = 5,
  • minSDK : The minSDK value from the manifest or 11 if not present.

  • versionCode : The versionCode from the manifest.

With these keys the user can define a pattern of

{abi}{minSDK}{versionCode}

or if they way to include zero padding they can use

{abi}{minSDK}{versionCode:D4}

similar to the left padding formats used in string.Format ().

Users can also use the $(AndroidVersionCodeProperties) property
to define new custom keys. This string will be in the form of a
semi-colon delimited key=value pairs. For example

foo=12;bar=$(SomeBuildProperty)

when can then be used in the pattern.

{abi}{foo}{bar}{versionCode}

Lets work through an example. The user defines a version code of '123'
in the manifest and enables $(AndroidCreatePackagePerAbi). They define
a $(AndroidVersionCodePattern) of {abi}{versionCode:D5}.
This will result in the following version code being produced for the
'x86' build.

300123

The first 3 is the {abi} value. The rest is the left zero padded
versionCode.
A slightly more complex pattern would be {abi}{minSDK:D2}{versionCode:D4}
which would produce

3140123

if the minimumSdk value was set to API 14.

A more real life example mgiht be as follows. A user wants to use the Build value
from the AssemblyInfo.cs . They define the following target

<Target Name="_GetBuild" AfterTargets="Compile">
  <GetAssemblyIdentity AssemblyFiles="Foo.dll">
      <Output
          TaskParameter="Assemblies"
          ItemName="MasterVersion"/>
    </GetAssemblyIdentity>
    <PropertyGroup>
       <BuildVersion>$([System.Version]::Parse(%(MasterVersion.Version)).Build)</BuildVersion>
    </PropertyGroup>
</Target>

This extracts the build version from the built assembly. They can then define a
pattern of

{abi}{minSDK}{build:D4}

and set the properties to

build=$(BuildVersion)

Given similar properties from the previous example e.g abi=x86 and minSDk=14,
this will result in the follwing output (assuming the Build value was 3421).

3143421

[1] https://developer.xamarin.com/guides/android/advanced_topics/build-abi-specific-apks/
[2] https://developer.android.com/google/play/publishing/multiple-apks.html#Rules

@dellis1972 dellis1972 added the do-not-merge PR should not be merged. label Jul 12, 2017
{
var regex = new Regex ("\\{(?<key>([A-Za-z]*)):?0*[\\}]");
var kvp = new Dictionary<string, int> ();
foreach (var item in versionCodeProperties?.Split (new char [] { ';' }) ?? new string [0]) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should split on both ; and :, or just :.

; can't be specified on the command-line as part of a property value, as it's the property separator:

msbuild /p:Foo=Bar;Baz=Qux


public void CalculateVersionCode (string currentAbi, string versionCodePattern, string versionCodeProperties)
{
var regex = new Regex ("\\{(?<key>([A-Za-z]*)):?0*[\\}]");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want (?<key>([A-Za-z]*))? That would match no characters, allowing a match of {}. Is that sensible?

Additionally, does the 0* make sense? That looks like it should be any format string, so what would be wrong with e.g. {versionCode:x8}?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After thinking a bit more I think we should only support integer format padding strings.. Since the version code MUST be an integer it makes no sense to support anything else.

So we should only support

{0:00000}
{0:D4}

for padding.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that there are multiple format strings which result in an integer, e.g. g, and I don't think that whitelisting is sane.

I'm also not sure that it matters, necessarily? If they specify e.g. x4 and get a letter in the resulting versionCode, then aapt/etc. will complain later about the invalid version. So long as the resulting error message is sensible, it shouldn't matter where it errors, so long as it does.

Some examples, if `abi` is `armeabi` and `versionCode` in the manifest
is `123`

`{abi}{versionCode}`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line shouldn't begin/end with ````` (backtick), unless you want the property to contain backticks...

msbuild '/p:AndroidVersionCodePattern=`{abi}{versionCode}`'

I assume you don't, so this line and the line ~10 lines down likewise shouldn't begin/end with backticks. These lines are already indented extra, which makes them formatted as code.

- **versionCode** &ndsh; Uses the version code direrctly from
`Properties\AndroidManifest.xml`.

You can define custom items using the [AndroidVersionCodeProperties] (#AndroidVersionCodeProperties)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be no space between the ] and the (.


Pre defined key items

- **abi** &ndsh; Inserts the targetted abi for the app
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be &ndash;, not &ndsh;; note the missing a.

- **AndroidVersionCodePattern** &ndsh; A string property which allows
the developer to customize the `versionCode` in the manifest when splitting
up the apk by abi.
See [Creating_the_Version_Code_for_the_APK](https://developer.xamarin.com/guides/android/advanced_topics/build-abi-specific-apks/#Creating_the_Version_Code_for_the_APK)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't contain _s:

See [Creating the Version Code for the APK](https://developer.xamarin.com/guides/android/advanced_topics/build-abi-specific-apks/#Creating_the_Version_Code_for_the_APK)


Added in Xamarin.Android 7.2.
- **AndroidVersionCodeProperties** &ndsh; A string property which allows
the developer to define custom items to use with the [AndroidVersionCodePattern] (#AndroidVersionCodePattern).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto here; no space between ] and (.

property.

Added in Xamarin.Android 7.2.
- **AndroidVersionCodeProperties** &ndsh; A string property which allows
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is documented, but I don't see any tests which exercise this property.

@dellis1972
Copy link
Contributor Author

dellis1972 commented Jul 13, 2017 via email

@dellis1972 dellis1972 force-pushed the versionCode branch 5 times, most recently from b07d47f to f1be8a7 Compare July 18, 2017 21:16
@dellis1972 dellis1972 removed the do-not-merge PR should not be merged. label Jul 19, 2017
public string MinimumSdk {
get {
var uses = doc.Root.Element ("uses-sdk");
if (uses?.Attribute (androidNs + "minSdkVersion") == null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that you actually want the minSdkVersion attribute value, why not look for that instead of the element?

var minAttr = doc.Root.Element ("uses-sdk")?.Attribute (androidNs + "minSdkVersion");
if (minAttr == null) {
    ...
}
return minAttr.Value;

@@ -68,6 +70,19 @@ internal class ManifestDocument
doc.Root.SetAttributeValue (androidNs + "versionCode", value);
}
}
public string MinimumSdk {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this should be a GetMinimumSdk() method, not a property, given that this property isn't just a field access? It would inform callers that they may want to cache the value instead of calling the property repeatedly.


{abi}{versionCode}

will prodice a versionCode of `1123`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be:

...will produce [sic] a versionCode of 11123 when $(AndroidCreatePackagePerAbi) is True, otherwise will produce a value of 123.


will prodice a versionCode of `1123`.
If `abi` is `x86_64` and `versionCode` in the manifest
is `44`. This will produce `544`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This similarly should mention $(AndroidCreatePackagePerAbi).

Added in Xamarin.Android 7.2.

- **AndroidVersionCodeProperties** &ndash; A string property which allows
the developer to define custom items to use with the [AndroidVersionCodePattern](#AndroidVersionCodePattern).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a prior documentation repro which moved this to be a list instead of a table broke the hyperlinks, or, hyperlinks now require post-processing/hosting on developer.xamarin.com

Specifically, this link doesn't go anywhere when simply rendered as HTML.

Perhaps this should add <a name="..." /> so that these links work?

@dellis1972 dellis1972 changed the title [Xamarin.Android.Build.Tasks] Implement a new process for defining versionCode. [WIP] [Xamarin.Android.Build.Tasks] Implement a new process for defining versionCode. Jul 26, 2017
…rsionCode.

Context https://bugzilla.xamarin.com/show_bug.cgi?id=51620
Context https://bugzilla.xamarin.com/show_bug.cgi?id=51618
Context https://bugzilla.xamarin.com/show_bug.cgi?id=51145

Our current version system for multiple apk's for each Abi
is a bit broken [1]. If a user for example has a versionCode
set which is 123 the final version code for an x86_64 build
ends up as 327803.
This is completely transparent to the user and also does not
follow the guidance in the documentation at [1] and [2].

So we need a new system :) but as usual we have to support the
old system. So we are introducing a new system which is more
flexible. This will only apply when the `$(AndroidCreatePackagePerAbi)`
is set to `True`.

The new system has two new properties

	<AndroidVersionCodePattern/>
	<AndroidVersionCodeProperties/>

The first allows the developer to define the Pattern to be used
for the versonCode. The pattern will be made up of a format string
which will contain keys. These keys will be replaced with values
form one of the known keys or a custom user defined one.

We define a few known key values

- abi : The current target abi converted to an int where
	- 'armeabi' = 1,
	- 'armeabi-v7a' = 2,
	- 'x86' = 3,
	- 'arm64-v8a' = 4,
	- 'x86_64' = 5,

- minSDK : The minSDK value from the manifest or 11 if not present.

- versionCode : The versionCode from the manifest.

With these keys the user can define a pattern of

	{abi}{minSDK}{versionCode}

or if they way to include zero padding they can use

	{abi}{minSDK}{versionCode:D4}

similar to the left padding formats used in string.Format ().

Users can also use the `$(AndroidVersionCodeProperties)` property
to define new custom keys. This string will be in the form of a
semi-colon delimited key=value pairs. For example

	foo=12;bar=$(SomeBuildProperty)

when can then be used in the pattern.

	{abi}{foo}{bar}{versionCode}

Lets work through an example. The user defines a version code of '123'
in the manifest and enables `$(AndroidCreatePackagePerAbi)`. They define
a `$(AndroidVersionCodePattern)` of `{abi}{versionCode:D5}`.
This will result in the following version code being produced for the
'x86' build.

	300123

The first 3 is the `{abi}` value. The rest is the left zero padded
versionCode.
A slightly more complex pattern would be `{abi}{minSDK:D2}{versionCode:D4}`
which would produce

	3140123

if the minimumSdk value was set to API 14.

A more real life example mgiht be as follows. A user wants to use the `Build` value
from the AssemblyInfo.cs . They define the following target

```xml
<Target Name="_GetBuild" AfterTargets="Compile">
  <GetAssemblyIdentity AssemblyFiles="Foo.dll">
      <Output
          TaskParameter="Assemblies"
          ItemName="MasterVersion"/>
    </GetAssemblyIdentity>
    <PropertyGroup>
       <BuildVersion>$([System.Version]::Parse(%(MasterVersion.Version)).Build)</BuildVersion>
    </PropertyGroup>
</Target>
```
This extracts the build version from the built assembly. They can then define a
pattern of

	{abi}{minSDK}{build:D4}

and set the properties to

	build=$(BuildVersion)

Given similar properties from the previous example e.g abi=x86 and minSDk=14,
this will result in the follwing output (assuming the `Build` value was 3421).

	3143421

[1] https://developer.xamarin.com/guides/android/advanced_topics/build-abi-specific-apks/
[2] https://developer.android.com/google/play/publishing/multiple-apks.html#Rules
@jonpryor jonpryor merged commit 0a2f008 into dotnet:master Jul 27, 2017
dellis1972 added a commit that referenced this pull request Aug 1, 2017
…rsionCode. (#692)

Context: https://bugzilla.xamarin.com/show_bug.cgi?id=51620
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=51618
Context: https://bugzilla.xamarin.com/show_bug.cgi?id=51145

Our current version system for multiple apk's for each Abi
is a bit broken [1]. If a user for example has a versionCode
set which is 123 the final version code for an x86_64 build
ends up as 327803.
This is completely transparent to the user and also does not
follow the guidance in the documentation at [1] and [2].

So we need a new system :) but as usual we have to support the
old system. So we are introducing a new system which is more
flexible. This will only apply when the `$(AndroidCreatePackagePerAbi)`
is set to `True`.

The new system has two new properties

	<AndroidVersionCodePattern/>
	<AndroidVersionCodeProperties/>

The first allows the developer to define the Pattern to be used
for the versonCode. The pattern will be made up of a format string
which will contain keys. These keys will be replaced with values
form one of the known keys or a custom user defined one.

We define a few known key values

- abi : The current target abi converted to an int where
	- 'armeabi' = 1,
	- 'armeabi-v7a' = 2,
	- 'x86' = 3,
	- 'arm64-v8a' = 4,
	- 'x86_64' = 5,

- minSDK : The minSDK value from the manifest or 11 if not present.

- versionCode : The versionCode from the manifest.

With these keys the user can define a pattern of

	{abi}{minSDK}{versionCode}

or if they way to include zero padding they can use

	{abi}{minSDK}{versionCode:D4}

similar to the left padding formats used in string.Format ().

Users can also use the `$(AndroidVersionCodeProperties)` property
to define new custom keys. This string will be in the form of a
semi-colon delimited key=value pairs. For example

	foo=12;bar=$(SomeBuildProperty)

when can then be used in the pattern.

	{abi}{foo}{bar}{versionCode}

Lets work through an example. The user defines a version code of '123'
in the manifest and enables `$(AndroidCreatePackagePerAbi)`. They define
a `$(AndroidVersionCodePattern)` of `{abi}{versionCode:D5}`.
This will result in the following version code being produced for the
'x86' build.

	300123

The first 3 is the `{abi}` value. The rest is the left zero padded
versionCode.
A slightly more complex pattern would be `{abi}{minSDK:D2}{versionCode:D4}`
which would produce

	3140123

if the minimumSdk value was set to API 14.

A more real life example mgiht be as follows. A user wants to use the `Build` value
from the AssemblyInfo.cs . They define the following target

```xml
<Target Name="_GetBuild" AfterTargets="Compile">
  <GetAssemblyIdentity AssemblyFiles="Foo.dll">
      <Output
          TaskParameter="Assemblies"
          ItemName="MasterVersion"/>
    </GetAssemblyIdentity>
    <PropertyGroup>
       <BuildVersion>$([System.Version]::Parse(%(MasterVersion.Version)).Build)</BuildVersion>
    </PropertyGroup>
</Target>
```
This extracts the build version from the built assembly. They can then define a
pattern of

	{abi}{minSDK}{build:D4}

and set the properties to

	build=$(BuildVersion)

Given similar properties from the previous example e.g abi=x86 and minSDk=14,
this will result in the follwing output (assuming the `Build` value was 3421).

	3143421

[1] https://developer.xamarin.com/guides/android/advanced_topics/build-abi-specific-apks/
[2] https://developer.android.com/google/play/publishing/multiple-apks.html#Rules
jonpryor added a commit to jonpryor/xamarin-android that referenced this pull request Aug 25, 2020
Fixes: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1139578

Context: dotnet/java-interop#691
Context: https://liquid.microsoft.com/Web/Object/Read/ms.security/Requirements/Microsoft.Security.SystemsADM.10039#guide

Changes: dotnet/java-interop@007b35b...08ff4db

  * dotnet/java-interop@08ff4db1: [java-interop, Java.Interop] Securely load native libs (dotnet#691)
  * dotnet/java-interop@09c68911: [generator] Clean up generated whitespace (dotnet#692)
  * dotnet/java-interop@d664e90e: [generator] Make warning and error messages localizable. (dotnet#689)

For improved security, the guidance is that when loading `.dll` files
on Windows, the default [`LoadLibrary()`][0] behavior is to be
*avoided*, as the [Dynamic-Link Library Search Order][1] includes the
"current working directory" and directories listed in `%PATH%`, both
of which are under control of an "attacker" and not the application.

Instead, [`LoadLibraryEx()`][2] should be used, providing a set of
`LOAD_LIBRARY_SEARCH_*` values which constrain the set of directories
that will be searched for the library (if a full path is not used)
and/or the library's dependencies.

Historically, Xamarin.Android hasn't directly called any
`LoadLibrary()` function.  Instead, xamarin-android called
[**dlopen**(3)][3], then used dlfcn-win32/dlfcn-win32@ef7e412d to
implement `dlopen()` in terms of `LoadLibraryEx()`.

Unfortunately, dlfcn-win32/dlfcn-win32@ef7e412d calls `LoadLibraryEx()`
with `LOAD_WITH_ALTERED_SEARCH_PATH`, which while a security
improvement, does not go far enough for our internal requirements.

Migrate away from dlfcn-win32 and instead use the new
`java-interop-dlfcn.h` family of functions added in
dotnet/java-interop@08ff4db1.  These new functions appropriately use
`LoadLibraryEx()` with a constrained set of `LOAD_LIBRARY_SEARCH_*`
flags.

*Non-obvious semantic note*: On Mono for all platforms, P/Invoking
`__Internal` is equivalent to trying to load the path `NULL`, e.g.
on macOS `dlopen(NULL, …)`.  However, only Android allows a custom
"`loader_func`" to intercept the attempted load of `NULL` and "do
something" with it, which xamarin-android *does* intercept, "remapping"
`NULL` to `libxa-internal-api.so`.  On all other platforms,
`loader_func` cannot intercept this invocation.

…which made for an "interesting" interaction: on macOS, if we used:

	api_dso_handle = java_interop_lib_load (dso_path.get (), JAVA_INTEROP_LIB_LOAD_LOCALLY, nullptr);

then the macOS Designer integration tests would fail!

	Renderer >> 4 [monodroid] Calling into managed runtime init
	Renderer (error) >>
	Renderer (error) >> Unhandled Exception:
	Renderer (error) >> System.EntryPointNotFoundException: java_interop_jnienv_get_java_vm assembly:<unknown assembly> type:<unknown type> member:(null)
	Renderer (error) >> at (wrapper managed-to-native) Java.Interop.NativeMethods.java_interop_jnienv_get_java_vm(intptr,intptr&)
	Renderer (error) >> at Java.Interop.JniEnvironment+References.GetJavaVM (System.IntPtr jnienv, System.IntPtr& vm) [0x00000] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Java.Interop.JniEnvironmentInfo.set_EnvironmentPointer (System.IntPtr value) [0x00037] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Java.Interop.JniEnvironmentInfo..ctor (System.IntPtr environmentPointer, Java.Interop.JniRuntime runtime) [0x00006] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Java.Interop.JniRuntime..ctor (Java.Interop.JniRuntime+CreationOptions options) [0x0017b] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Android.Runtime.AndroidRuntime..ctor (System.IntPtr jnienv, System.IntPtr vm, System.Boolean allocNewObjectSupported, System.IntPtr classLoader, System.IntPtr classLoader_loadClass, System.Boolean jniAddNativeMethodRegistrationAttributePresent) [0x00000] in /Users/builder/azdo/_work/4/s/xamarin-android/src/Mono.Android/Android.Runtime/AndroidRuntime.cs:25

In order for the macOS Designer integration tests to succeed,
`libxa-internal-api.dylib` *must* be loaded as `RTLD_GLOBAL`, which is
done via `JAVA_INTEROP_LIB_LOAD_GLOBALLY`.

[0]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibrarya
[1]: https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order
[2]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexa
[3]: https://linux.die.net/man/3/dlopen
jonpryor added a commit to jonpryor/xamarin-android that referenced this pull request Aug 26, 2020
Fixes: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1139578

Context: dotnet/java-interop#691
Context: https://liquid.microsoft.com/Web/Object/Read/ms.security/Requirements/Microsoft.Security.SystemsADM.10039#guide

Changes: dotnet/java-interop@007b35b...08ff4db

  * dotnet/java-interop@08ff4db1: [java-interop, Java.Interop] Securely load native libs (dotnet#691)
  * dotnet/java-interop@09c68911: [generator] Clean up generated whitespace (dotnet#692)
  * dotnet/java-interop@d664e90e: [generator] Make warning and error messages localizable. (dotnet#689)

For improved security, the guidance is that when loading `.dll` files
on Windows, the default [`LoadLibrary()`][0] behavior is to be
*avoided*, as the [Dynamic-Link Library Search Order][1] includes the
"current working directory" and directories listed in `%PATH%`, both
of which are under control of an "attacker" and not the application.

Instead, [`LoadLibraryEx()`][2] should be used, providing a set of
`LOAD_LIBRARY_SEARCH_*` values which constrain the set of directories
that will be searched for the library (if a full path is not used)
and/or the library's dependencies.

Historically, Xamarin.Android hasn't directly called any
`LoadLibrary()` function.  Instead, xamarin-android called
[**dlopen**(3)][3], then used dlfcn-win32/dlfcn-win32@ef7e412d to
implement `dlopen()` in terms of `LoadLibraryEx()`.

Unfortunately, dlfcn-win32/dlfcn-win32@ef7e412d calls `LoadLibraryEx()`
with `LOAD_WITH_ALTERED_SEARCH_PATH`, which while a security
improvement, does not go far enough for our internal requirements.

Migrate away from dlfcn-win32 and instead use the new
`java-interop-dlfcn.h` family of functions added in
dotnet/java-interop@08ff4db1.  These new functions appropriately use
`LoadLibraryEx()` with a constrained set of `LOAD_LIBRARY_SEARCH_*`
flags.

*Non-obvious semantic note*: On Mono for all platforms, P/Invoking
`__Internal` is equivalent to trying to load the path `NULL`, e.g.
on macOS `dlopen(NULL, …)`.  However, only Android allows a custom
"`loader_func`" to intercept the attempted load of `NULL` and "do
something" with it, which xamarin-android *does* intercept, "remapping"
`NULL` to `libxa-internal-api.so`.  On all other platforms,
`loader_func` cannot intercept this invocation.

…which made for an "interesting" interaction: on macOS, if we used:

	api_dso_handle = java_interop_lib_load (dso_path.get (), JAVA_INTEROP_LIB_LOAD_LOCALLY, nullptr);

then the macOS Designer integration tests would fail!

	Renderer >> 4 [monodroid] Calling into managed runtime init
	Renderer (error) >>
	Renderer (error) >> Unhandled Exception:
	Renderer (error) >> System.EntryPointNotFoundException: java_interop_jnienv_get_java_vm assembly:<unknown assembly> type:<unknown type> member:(null)
	Renderer (error) >> at (wrapper managed-to-native) Java.Interop.NativeMethods.java_interop_jnienv_get_java_vm(intptr,intptr&)
	Renderer (error) >> at Java.Interop.JniEnvironment+References.GetJavaVM (System.IntPtr jnienv, System.IntPtr& vm) [0x00000] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Java.Interop.JniEnvironmentInfo.set_EnvironmentPointer (System.IntPtr value) [0x00037] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Java.Interop.JniEnvironmentInfo..ctor (System.IntPtr environmentPointer, Java.Interop.JniRuntime runtime) [0x00006] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Java.Interop.JniRuntime..ctor (Java.Interop.JniRuntime+CreationOptions options) [0x0017b] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Android.Runtime.AndroidRuntime..ctor (System.IntPtr jnienv, System.IntPtr vm, System.Boolean allocNewObjectSupported, System.IntPtr classLoader, System.IntPtr classLoader_loadClass, System.Boolean jniAddNativeMethodRegistrationAttributePresent) [0x00000] in /Users/builder/azdo/_work/4/s/xamarin-android/src/Mono.Android/Android.Runtime/AndroidRuntime.cs:25

In order for the macOS Designer integration tests to succeed,
`libxa-internal-api.dylib` *must* be loaded as `RTLD_GLOBAL`, which is
done via `JAVA_INTEROP_LIB_LOAD_GLOBALLY`.

[0]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibrarya
[1]: https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order
[2]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexa
[3]: https://linux.die.net/man/3/dlopen
jonpryor added a commit to jonpryor/xamarin-android that referenced this pull request Aug 26, 2020
Fixes: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1139578

Context: dotnet/java-interop#691
Context: https://liquid.microsoft.com/Web/Object/Read/ms.security/Requirements/Microsoft.Security.SystemsADM.10039#guide

Changes: dotnet/java-interop@007b35b...dcaf794

  * dotnet/java-interop@dcaf794d: [generator] Fix error and warning codes values (dotnet#697)
  * dotnet/java-interop@08ff4db1: [java-interop, Java.Interop] Securely load native libs (dotnet#691)
  * dotnet/java-interop@09c68911: [generator] Clean up generated whitespace (dotnet#692)
  * dotnet/java-interop@d664e90e: [generator] Make warning and error messages localizable. (dotnet#689)

For improved security, the guidance is that when loading `.dll` files
on Windows, the default [`LoadLibrary()`][0] behavior is to be
*avoided*, as the [Dynamic-Link Library Search Order][1] includes the
"current working directory" and directories listed in `%PATH%`, both
of which are under control of an "attacker" and not the application.

Instead, [`LoadLibraryEx()`][2] should be used, providing a set of
`LOAD_LIBRARY_SEARCH_*` values which constrain the set of directories
that will be searched for the library (if a full path is not used)
and/or the library's dependencies.

Historically, Xamarin.Android hasn't directly called any
`LoadLibrary()` function.  Instead, xamarin-android called
[**dlopen**(3)][3], then used dlfcn-win32/dlfcn-win32@ef7e412d to
implement `dlopen()` in terms of `LoadLibraryEx()`.

Unfortunately, dlfcn-win32/dlfcn-win32@ef7e412d calls `LoadLibraryEx()`
with `LOAD_WITH_ALTERED_SEARCH_PATH`, which while a security
improvement, does not go far enough for our internal requirements.

Migrate away from dlfcn-win32 and instead use the new
`java-interop-dlfcn.h` family of functions added in
dotnet/java-interop@08ff4db1.  These new functions appropriately use
`LoadLibraryEx()` with a constrained set of `LOAD_LIBRARY_SEARCH_*`
flags.

*Non-obvious semantic note*: On Mono for all platforms, P/Invoking
`__Internal` is equivalent to trying to load the path `NULL`, e.g.
on macOS `dlopen(NULL, …)`.  However, only Android allows a custom
"`loader_func`" to intercept the attempted load of `NULL` and "do
something" with it, which xamarin-android *does* intercept, "remapping"
`NULL` to `libxa-internal-api.so`.  On all other platforms,
`loader_func` cannot intercept this invocation.

…which made for an "interesting" interaction: on macOS, if we used:

	api_dso_handle = java_interop_lib_load (dso_path.get (), JAVA_INTEROP_LIB_LOAD_LOCALLY, nullptr);

then the macOS Designer integration tests would fail!

	Renderer >> 4 [monodroid] Calling into managed runtime init
	Renderer (error) >>
	Renderer (error) >> Unhandled Exception:
	Renderer (error) >> System.EntryPointNotFoundException: java_interop_jnienv_get_java_vm assembly:<unknown assembly> type:<unknown type> member:(null)
	Renderer (error) >> at (wrapper managed-to-native) Java.Interop.NativeMethods.java_interop_jnienv_get_java_vm(intptr,intptr&)
	Renderer (error) >> at Java.Interop.JniEnvironment+References.GetJavaVM (System.IntPtr jnienv, System.IntPtr& vm) [0x00000] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Java.Interop.JniEnvironmentInfo.set_EnvironmentPointer (System.IntPtr value) [0x00037] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Java.Interop.JniEnvironmentInfo..ctor (System.IntPtr environmentPointer, Java.Interop.JniRuntime runtime) [0x00006] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Java.Interop.JniRuntime..ctor (Java.Interop.JniRuntime+CreationOptions options) [0x0017b] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Android.Runtime.AndroidRuntime..ctor (System.IntPtr jnienv, System.IntPtr vm, System.Boolean allocNewObjectSupported, System.IntPtr classLoader, System.IntPtr classLoader_loadClass, System.Boolean jniAddNativeMethodRegistrationAttributePresent) [0x00000] in /Users/builder/azdo/_work/4/s/xamarin-android/src/Mono.Android/Android.Runtime/AndroidRuntime.cs:25

In order for the macOS Designer integration tests to succeed,
`libxa-internal-api.dylib` *must* be loaded as `RTLD_GLOBAL`, which is
done via `JAVA_INTEROP_LIB_LOAD_GLOBALLY`.

[0]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibrarya
[1]: https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order
[2]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexa
[3]: https://linux.die.net/man/3/dlopen
jonpryor added a commit that referenced this pull request Aug 27, 2020
Fixes: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1139578

Context: dotnet/java-interop#691
Context: https://liquid.microsoft.com/Web/Object/Read/ms.security/Requirements/Microsoft.Security.SystemsADM.10039#guide

Changes: dotnet/java-interop@007b35b...dcaf794

  * dotnet/java-interop@dcaf794d: [generator] Fix error and warning codes values (#697)
  * dotnet/java-interop@08ff4db1: [java-interop, Java.Interop] Securely load native libs (#691)
  * dotnet/java-interop@09c68911: [generator] Clean up generated whitespace (#692)
  * dotnet/java-interop@d664e90e: [generator] Make warning and error messages localizable. (#689)

For improved security, the guidance is that when loading `.dll` files
on Windows, the default [`LoadLibrary()`][0] behavior is to be
*avoided*, as the [Dynamic-Link Library Search Order][1] includes the
"current working directory" and directories listed in `%PATH%`, both
of which are under control of an "attacker" and not the application.

Instead, [`LoadLibraryEx()`][2] should be used, providing a set of
`LOAD_LIBRARY_SEARCH_*` values which constrain the set of directories
that will be searched for the library (if a full path is not used)
and/or the library's dependencies.

Historically, Xamarin.Android hasn't directly called any
`LoadLibrary()` function.  Instead, xamarin-android called
[**dlopen**(3)][3], then used dlfcn-win32/dlfcn-win32@ef7e412d to
implement `dlopen()` in terms of `LoadLibraryEx()`.

Unfortunately, dlfcn-win32/dlfcn-win32@ef7e412d calls `LoadLibraryEx()`
with `LOAD_WITH_ALTERED_SEARCH_PATH`, which while a security
improvement, does not go far enough for our internal requirements.

Migrate away from dlfcn-win32 and instead use the new
`java-interop-dlfcn.h` family of functions added in
dotnet/java-interop@08ff4db1.  These new functions appropriately use
`LoadLibraryEx()` with a constrained set of `LOAD_LIBRARY_SEARCH_*`
flags.

*Non-obvious semantic note*: On Mono for all platforms, P/Invoking
`__Internal` is equivalent to trying to load the path `NULL`, e.g.
on macOS `dlopen(NULL, …)`.  However, only Android allows a custom
"`loader_func`" to intercept the attempted load of `NULL` and "do
something" with it, which xamarin-android *does* intercept, "remapping"
`NULL` to `libxa-internal-api.so`.  On all other platforms,
`loader_func` cannot intercept this invocation.

…which made for an "interesting" interaction: on macOS, if we used:

	api_dso_handle = java_interop_lib_load (dso_path.get (), JAVA_INTEROP_LIB_LOAD_LOCALLY, nullptr);

then the macOS Designer integration tests would fail!

	Renderer >> 4 [monodroid] Calling into managed runtime init
	Renderer (error) >>
	Renderer (error) >> Unhandled Exception:
	Renderer (error) >> System.EntryPointNotFoundException: java_interop_jnienv_get_java_vm assembly:<unknown assembly> type:<unknown type> member:(null)
	Renderer (error) >> at (wrapper managed-to-native) Java.Interop.NativeMethods.java_interop_jnienv_get_java_vm(intptr,intptr&)
	Renderer (error) >> at Java.Interop.JniEnvironment+References.GetJavaVM (System.IntPtr jnienv, System.IntPtr& vm) [0x00000] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Java.Interop.JniEnvironmentInfo.set_EnvironmentPointer (System.IntPtr value) [0x00037] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Java.Interop.JniEnvironmentInfo..ctor (System.IntPtr environmentPointer, Java.Interop.JniRuntime runtime) [0x00006] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Java.Interop.JniRuntime..ctor (Java.Interop.JniRuntime+CreationOptions options) [0x0017b] in <0f003a4904fd44d0a8cc6a63962ab40b>:0
	Renderer (error) >> at Android.Runtime.AndroidRuntime..ctor (System.IntPtr jnienv, System.IntPtr vm, System.Boolean allocNewObjectSupported, System.IntPtr classLoader, System.IntPtr classLoader_loadClass, System.Boolean jniAddNativeMethodRegistrationAttributePresent) [0x00000] in /Users/builder/azdo/_work/4/s/xamarin-android/src/Mono.Android/Android.Runtime/AndroidRuntime.cs:25

In order for the macOS Designer integration tests to succeed,
`libxa-internal-api.dylib` *must* be loaded as `RTLD_GLOBAL`, which is
done via `JAVA_INTEROP_LIB_LOAD_GLOBALLY`.

[0]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibrarya
[1]: https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order
[2]: https://docs.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexa
[3]: https://linux.die.net/man/3/dlopen
@github-actions github-actions bot locked and limited conversation to collaborators Feb 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants