Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Mono.Android] Bind Android 12L Beta 1. #6601

Merged
merged 18 commits into from
Jan 14, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions Configuration.Override.props.in
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@
kept consistent with each other, lest Bad Things Happen™
-->
<!-- The default Android API level to bind in src/Mono.Android -->
<AndroidApiLevel>31</AndroidApiLevel>
<AndroidApiLevel>32</AndroidApiLevel>
<!-- The Xamarin.Android $(TargetFrameworkVersion) value that corresponds to $(AndroidApiLevel) -->
<AndroidFrameworkVersion>v12.0</AndroidFrameworkVersion>
<AndroidFrameworkVersion>v12.1</AndroidFrameworkVersion>
<!-- The default Android API "id" that corresponds to $(AndroidApiLevel) -->
<AndroidPlatformId>31</AndroidPlatformId>
<AndroidPlatformId>32</AndroidPlatformId>

<!--
Colon-separated list of ABIs to build the mono JIT for.
Expand Down
28 changes: 19 additions & 9 deletions Configuration.props
Original file line number Diff line number Diff line change
Expand Up @@ -26,17 +26,19 @@
<AndroidFirstFrameworkVersion Condition="'$(AndroidFirstFrameworkVersion)' == ''">v4.4</AndroidFirstFrameworkVersion>
<AndroidFirstApiLevel Condition="'$(AndroidFirstApiLevel)' == ''">19</AndroidFirstApiLevel>
<AndroidJavaRuntimeApiLevel Condition="'$(AndroidJavaRuntimeApiLevel)' == ''">21</AndroidJavaRuntimeApiLevel>
<!-- The default API level used for $(TargetPlatformVersion) -->
<AndroidDefaultTargetDotnetApiLevel Condition="'$(AndroidDefaultTargetDotnetApiLevel)' == ''">31</AndroidDefaultTargetDotnetApiLevel>
<!-- The min API level supported by Microsoft.Android.Sdk, should refactor/remove when this value is the same as $(AndroidFirstApiLevel) -->
<AndroidMinimumDotNetApiLevel Condition="'$(AndroidMinimumDotNetApiLevel)' == ''">21</AndroidMinimumDotNetApiLevel>
<AndroidFirstPlatformId Condition="'$(AndroidFirstPlatformId)' == ''">$(AndroidFirstApiLevel)</AndroidFirstPlatformId>
<!-- *Latest* *stable* API level binding that we support; used when building src/Xamarin.Android.Build.Tasks -->
<AndroidLatestStableApiLevel Condition="'$(AndroidLatestStableApiLevel)' == ''">31</AndroidLatestStableApiLevel>
<AndroidLatestStableApiLevel Condition="'$(AndroidLatestStableApiLevel)' == ''">32</AndroidLatestStableApiLevel>
<AndroidLatestStablePlatformId Condition="'$(AndroidLatestStablePlatformId)' == ''">$(AndroidLatestStableApiLevel)</AndroidLatestStablePlatformId>
<AndroidLatestStableFrameworkVersion Condition="'$(AndroidLatestStableFrameworkVersion)'==''">v12.0</AndroidLatestStableFrameworkVersion>
<AndroidLatestStableFrameworkVersion Condition="'$(AndroidLatestStableFrameworkVersion)'==''">v12.1</AndroidLatestStableFrameworkVersion>
<!-- *Latest* *unstable* API level binding that we support; this can be the same as *stable* -->
<AndroidLatestUnstableApiLevel Condition="'$(AndroidLatestUnstableApiLevel)' == ''">32</AndroidLatestUnstableApiLevel>
<AndroidLatestUnstablePlatformId Condition="'$(AndroidLatestUnstablePlatformId)' == ''">Sv2</AndroidLatestUnstablePlatformId>
<AndroidLatestUnstableFrameworkVersion Condition="'$(AndroidLatestUnstableFrameworkVersion)'==''">v12.0.99</AndroidLatestUnstableFrameworkVersion>
<AndroidLatestUnstablePlatformId Condition="'$(AndroidLatestUnstablePlatformId)' == ''">32</AndroidLatestUnstablePlatformId>
<AndroidLatestUnstableFrameworkVersion Condition="'$(AndroidLatestUnstableFrameworkVersion)'==''">v12.1</AndroidLatestUnstableFrameworkVersion>
<!-- The API level and TargetFrameworkVersion for the default Mono.Android.dll build -->
<AndroidApiLevel Condition=" '$(AndroidApiLevel)' == '' ">$(AndroidLatestStableApiLevel)</AndroidApiLevel>
<AndroidPlatformId Condition=" '$(AndroidPlatformId)' == '' ">$(AndroidLatestStablePlatformId)</AndroidPlatformId>
Expand Down Expand Up @@ -118,16 +120,24 @@
<AllSupported64BitTargetAndroidAbis>arm64-v8a;x86_64</AllSupported64BitTargetAndroidAbis>
<AllSupportedTargetAndroidAbis>$(AllSupported32BitTargetAndroidAbis);$(AllSupported64BitTargetAndroidAbis)</AllSupportedTargetAndroidAbis>
<!--
For some reason, the URL for platform-tools/build-tools 30.0.3 is prefixed with what appears to be a GIT commit hash or some other checksum...
For some reason, the URL for platform-tools/build-tools 32.0.0 is prefixed with what appears to be a GIT commit hash or some other checksum...
Linux packages don't have any prefix, but this forces us to have *some* mechanism to handle this...
-->
<XABuildToolsPackagePrefixMacOS>f6d24b187cc6bd534c6c37604205171784ac5621.</XABuildToolsPackagePrefixMacOS>
<XABuildToolsPackagePrefixWindows>91936d4ee3ccc839f0addd53c9ebf087b1e39251.</XABuildToolsPackagePrefixWindows>
<XABuildToolsPackagePrefixMacOS>5219cc671e844de73762e969ace287c29d2e14cd.</XABuildToolsPackagePrefixMacOS>
<XABuildToolsPackagePrefixWindows>210b77e4bc623bd4cdda4dae790048f227972bd2.</XABuildToolsPackagePrefixWindows>
<XABuildToolsPackagePrefixLinux></XABuildToolsPackagePrefixLinux>
<XABuildToolsPackagePrefix Condition=" '$(HostOS)' == 'Darwin' ">$(XABuildToolsPackagePrefixMacOS)</XABuildToolsPackagePrefix>
<XABuildToolsPackagePrefix Condition=" '$(HostOS)' == 'Windows' ">$(XABuildToolsPackagePrefixWindows)</XABuildToolsPackagePrefix>
<XABuildToolsVersion>30.0.3</XABuildToolsVersion>
<XABuildToolsFolder Condition="'$(XABuildToolsFolder)' == ''">30.0.3</XABuildToolsFolder>
<XABuildToolsVersion>32</XABuildToolsVersion>
<XABuildToolsFolder Condition="'$(XABuildToolsFolder)' == ''">32.0.0</XABuildToolsFolder>
<!-- build-tools 30, for DX tests -->
<XABuildTools30PackagePrefixMacOS>f6d24b187cc6bd534c6c37604205171784ac5621.</XABuildTools30PackagePrefixMacOS>
<XABuildTools30PackagePrefixWindows>91936d4ee3ccc839f0addd53c9ebf087b1e39251.</XABuildTools30PackagePrefixWindows>
<XABuildTools30PackagePrefixLinux></XABuildTools30PackagePrefixLinux>
<XABuildTools30PackagePrefix Condition=" '$(HostOS)' == 'Darwin' ">$(XABuildTools30PackagePrefixMacOS)</XABuildTools30PackagePrefix>
<XABuildTools30PackagePrefix Condition=" '$(HostOS)' == 'Windows' ">$(XABuildTools30PackagePrefixWindows)</XABuildTools30PackagePrefix>
<XABuildTools30Version>30.0.3</XABuildTools30Version>
<XABuildTools30Folder Condition="'$(XABuildTools30Folder)' == ''">30.0.3</XABuildTools30Folder>
<XAPlatformToolsPackagePrefix Condition=" '$(HostOS)' == 'Darwin' ">e8b2b4cbe47c728c1e54c5f524440b52d4e1a33c.</XAPlatformToolsPackagePrefix>
<XAPlatformToolsVersion>31.0.3</XAPlatformToolsVersion>
<XAIncludeProprietaryBits Condition="'$(XAIncludeProprietaryBits)' == ''">False</XAIncludeProprietaryBits>
Expand Down
169 changes: 165 additions & 4 deletions Documentation/workflow/HowToAddNewApiLevel.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,174 @@ acceptable "breaks":

When Google announces that the APIs are frozen, additional work such as enumification is needed.

---- Somewhat outdated docs below, update when we do this year's stabilization ----
There have been many, many attempts to "automate" the enumification process in the past, to varying
degrees of success. The main problem is that no automated process is going to be perfect, so
they all rely on a human verifying and modifying the results.

However this verification process is long and tedious. Doing it correctly requires almost as much
effort as doing the full process manually. Thus it generally isn't done correctly and many errors
slip in, leaving our users with bad bindings that are hard to fix in the future without breaking API.

Currently we have taken the opposite approach and do the process completely manually, but we
have invested in tooling to make the process as easy as possible.

This tooling is BindingStudio:
https://github.com/jpobst/BindingStudio

It's a Winforms app, so it only runs on Windows. It's ugly as sin, and has very poor UX. However,
it prompts you with the exact decisions you need to make, and handles as much dirty work as possible,
allowing enumification to be done in a few days.

### Extract constants from API

Using BindingStudio:

- Update `CURRENT_API_LEVEL` in MainForm.cs
- Choose `Tools` -> `Add API Level Constants`
- Fill in existing `map.csv`: `xamarin-android/src/Mono.Android/map.csv`
- Fill in new `api.xml`: ex: `xamarin-android/src/Mono.Android/obj/Debug/net6.0/android-32/mcw/api.xml`
- Choose `File` -> `Save`

This adds all the new possible constants from the API level to `map.csv`. They will be
marked with a `?` indicating no determination has been made if they should be enumified or not.

Example:
```
?,32,android/media/Spatializer.SPATIALIZER_IMMERSIVE_LEVEL_MULTICHANNEL,1,,,,
?,32,android/media/Spatializer.SPATIALIZER_IMMERSIVE_LEVEL_NONE,0,,,,
?,32,android/media/Spatializer.SPATIALIZER_IMMERSIVE_LEVEL_OTHER,-1,,,,
```

### Creating enums

Using BindingStudio:

- Choose `File` -> `Open Constant Map`
- Choose existing `map.csv`: `xamarin-android/src/Mono.Android/map.csv`

The left tree view will be populated with every type that has possible constants that require
a decision. Clicking a tree node will show the grid of all constants in the type. The ones
that need to be handled are the ones with `Action` = `None`. (The others are shown because
sometimes the correct action is to add a new constant to an existing enum.)

Select the row(s) containing the constants you want to act on. (Use Control and Shift to select
multiple rows.) There are 3 possible options for constants:

1) Ignore

If the constant(s) should not be part of an enum (like `Math.PI`), click the `Ignore` toolbar
button to leave them as constants.

2) Add to existing enum

If the constant(s) should be added to an existing enum:
- Click the `Add to existing enum` toolbar button.
- The dialog will show all other enums in this type
- Choose the existing enum to add the new constant(s) to
- After accepting the dialog, you may need to click the grid to cause it to refresh
- The constant(s) will be marked as `Enumify` with the `EnumFullType` you specified
- The enum member names may need to be tweaked by changing the `EnumMember` column

3) Create a new enum

If the constant(s) should be added to a brand new enum:
- Click the `Create Enum` toolbar button
- In the dialog, a suggested enum namespace and name will be pre-populated. This may need to be
tweaked as needed.
- Mark `Is Flags` if this should be a `[Flags]` enum type.
- After accepting the dialog, you may need to click the grid to cause it to refresh
- The constant(s) will be marked as `Enumify` with the `EnumFullType` you specified
- The enum member names may need to be tweaked by changing the `EnumMember` column

Once decisions have been made for all new constants in a type, use the left tree view to move
to the next type. You should periodically save your progress with `File` -> `Save` in case
BindingStudio crashes.

The left tree view can be updated by saving and reopening the `map.csv` file.

### Extract methods that possibly need enums

5) enumification
Using BindingStudio:

See `build-tools/enumification-helpers/README`. Usually it takes many days to complete...
- Update the file paths in `MainForm.FindAPILevelMethodsToolStripMenuItem_Click`
- Run BindingStudio and choose `Tools` -> `Find API Level Methods`

Enumification work can be delayed and only the final API has to be enumified.
This will create a file of every method in the new API level that takes an `int` as a parameter
or returns an `int` as a return value. Each method will be marked with a `?` in the file
to indicate a decision needs to be made to ignore it or map it to an enum.

Example:
```
?,32,android/media,AudioAttributes,getSpatializationBehavior,return,
?,32,android/media,AudioAttributes$Builder,setSpatializationBehavior,sb,
```

### Mapping methods

Using BindingStudio:

- Choose `File` -> `Open Constant Map`
- Choose existing `map.csv`: `xamarin-android/src/Mono.Android/map.csv`
- Choose `File` -> `Open Method Map`
- Choose the new `.csv` created in the previous step

The left tree will populate with every method that possibly should be enumified and
needs a decision to be made. Clicking a method shows the Android documentation for
the method to help make the decision, as well as an area to input the decision.

Note a method may show up multiple times, once for each parameter or return type
(Parameter Name = "return") that is an int. Each one may require a different action.

There are 3 possible options for a method parameter/return type:

1) Unknown

You don't how to handle this method currently, so leaving it in the initial state
of "Unknown" will leave it alone until a decision can be made.

2) Ignore

The method parameter/return type should remain an `int` and not be converted to an enum.

Ex:
```
int Add (int value1, int value2) { ... }
```

Click the "Ignore" radio button and then the "Save" button.

3) Enumify

The method parameter/return type should be changed to an enum.

Ex:
```
void AudioAttributesBuilder.SetSpatializationBehavior (int sb) { ... }
```

- Choose the "Enumify" radio option
- Use the DropDown in the middle to select the enum to use
- When selected, the members of that enum will be shown in the box below the enum
- Alternatively, search for a enum by enum member name using the Search box in the right
- If desired enum is found, clicking it will populate dropdown
- Click "Save"

Use `File` -> `Save` to save your work often!

### Finishing the method map

The official `methodmap.csv` uses a slightly different format than the one used for enumification.

Using BindingStudio:
- Ensure the "new api level method map" CSV file is loaded.
- Choose `Tools` -> `Export Final Method Map`
- Choose a temporary file name
- Open the temporary file, copy the contents to the bottom of the official:
- xamarin-android/src/Mono.Android/methodmap.csv

Congrats! Enumification is complete!

---- Somewhat outdated docs below, update when we do this year's stabilization ----

6) new AndroidManifest.xml elements and attributes

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ public sealed class CheckApiCompatibility : Task
{ "v10.0", "v9.0" },
{ "v11.0", "v10.0" },
{ "v12.0", "v11.0" },
{ "v12.0.99", "v12.0" },
{ "v12.1", "v12.0" },
};

static readonly string assemblyToValidate = "Mono.Android.dll";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,15 @@ public class GenerateSupportedPlatforms : Task
[Required]
public int MinimumApiLevel { get; set; }

/// <summary>
/// Default value for $(TargetPlatformVersion), defaults to MaxStableVersion.ApiLevel
/// </summary>
public int TargetApiLevel { get; set; }

public override bool Execute ()
{
var versions = new AndroidVersions (AndroidApiInfo.Select (ToVersion));
int targetApiLevel = TargetApiLevel > 0 ? TargetApiLevel : versions.MaxStableVersion.ApiLevel;
var settings = new XmlWriterSettings {
OmitXmlDeclaration = true,
Indent = true,
Expand All @@ -56,7 +62,7 @@ public override bool Execute ()
writer.WriteEndElement (); // </TargetPlatformSupported>
writer.WriteStartElement ("TargetPlatformVersion");
writer.WriteAttributeString ("Condition", " '$(TargetPlatformVersion)' == '' ");
writer.WriteString (versions.MaxStableVersion.ApiLevel.ToString ("0.0", CultureInfo.InvariantCulture));
writer.WriteString (targetApiLevel.ToString ("0.0", CultureInfo.InvariantCulture));
writer.WriteEndElement (); // </TargetPlatformVersion>
writer.WriteStartElement ("AndroidMinimumSupportedApiLevel");
writer.WriteAttributeString ("Condition", " '$(AndroidMinimumSupportedApiLevel)' == '' ");
Expand Down
4 changes: 2 additions & 2 deletions build-tools/api-merge/merge-configuration.xml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
<File Path="api-29.xml.in" Level="29" />
<File Path="api-30.xml.in" Level="30" />
<File Path="api-31.xml.in" Level="31" />
<File Path="api-Sv2.xml.in" Level="Sv2" />
<File Path="api-32.xml.in" Level="32" />
</Inputs>
<Outputs>
<File Path="android-19\mcw\api.xml" LastLevel="19" />
Expand All @@ -36,6 +36,6 @@
<File Path="android-29\mcw\api.xml" LastLevel="29" />
<File Path="android-30\mcw\api.xml" LastLevel="30" />
<File Path="android-31\mcw\api.xml" LastLevel="31" />
<File Path="android-Sv2\mcw\api.xml" LastLevel="Sv2" />
<File Path="android-32\mcw\api.xml" LastLevel="32" />
</Outputs>
</Configuration>
2 changes: 1 addition & 1 deletion build-tools/api-xml-adjuster/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ API_XML_TOOL = $(BUILDBIN)/api-xml-adjuster.exe
RUNTIME = mono --debug
RUN_CLASS_PARSE = $(RUNTIME) $(CLASS_PARSE)
RUN_API_XML_TOOL = $(RUNTIME) $(API_XML_TOOL)
API_LEVELS = 10 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Sv2
API_LEVELS = 10 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

XML_OUTPUT_DIR = .

Expand Down
2 changes: 2 additions & 0 deletions build-tools/create-packs/Directory.Build.targets
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,8 @@
<_NuGetSources Include="$(OutputPath.TrimEnd('\'))" />
<_PreviewPacks Condition=" '$(AndroidLatestStableApiLevel)' != '$(AndroidLatestUnstableApiLevel)' " Include="$(XamarinAndroidSourcePath)bin\Build$(Configuration)\nuget-unsigned\Microsoft.Android.Ref.$(AndroidLatestUnstableApiLevel).*.nupkg" />
<_InstallArguments Include="android-aot" />
<!-- NOTE: temporary if $(AndroidDefaultTargetDotnetApiLevel) is not 32 -->
<_InstallArguments Include="android-32" />
<_InstallArguments Include="android-$(AndroidLatestUnstableApiLevel)" Condition=" '@(_PreviewPacks->Count())' != '0' " />
<_InstallArguments Include="--skip-manifest-update" />
<_InstallArguments Include="--verbosity diag" />
Expand Down
15 changes: 10 additions & 5 deletions build-tools/create-packs/Microsoft.Android.Sdk.proj
Original file line number Diff line number Diff line change
Expand Up @@ -118,18 +118,23 @@ WARNING: DO NOT MODIFY this file unless you are knowledgeable about MSBuild and
<PropertyGroup>
<AndroidNETSdkVersion>$(AndroidPackVersionLong)</AndroidNETSdkVersion>
<XamarinAndroidVersion>$(AndroidPackVersionLong)</XamarinAndroidVersion>
<_AndroidRuntimePackId Condition=" '%24(TargetPlatformVersion)' != '$(AndroidLatestUnstableApiLevel).0' ">$(AndroidLatestStableApiLevel)</_AndroidRuntimePackId>
<_AndroidRuntimePackId Condition=" '%24(TargetPlatformVersion)' == '$(AndroidLatestUnstableApiLevel).0' ">$(AndroidLatestUnstableApiLevel)</_AndroidRuntimePackId>
</PropertyGroup>
<PropertyGroup Condition=" '%24(TargetPlatformVersion)' == '31.0' ">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be using $(AndroidTargetDotNetApiLevel) (and possibly other new properties)? Isn't that what $(AndroidTargetDotNetApiLevel) is for?

Related: instead of $(AndroidTargetDotNetApiLevel), should we call it $(AndroidDefaultTargetDotnetApiLevel)? This "emphasizes" the "default" nature of this value.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In some of the other tests here, I'll try to use $(AndroidDefaultTargetDotnetApiLevel) so we don't have to change them again later.

In this case, however, the number 31 needs to remain, it will need to eternally pull this version from NuGet (or our workload):

https://github.com/xamarin/xamarin-android/blob/bedecc25f93a82ecaa8df5cb4e92424aa669125d/build-tools/create-packs/Microsoft.Android.Sdk.proj#L122-L125

I guess that over time, we'll have numbers listed for 32, 33, and so on. We can also update those packs at some point if ever needed (hopefully not ever?), we'd just list a different version number here.

<_AndroidRuntimePackId>31</_AndroidRuntimePackId>
<_AndroidRuntimePackVersion>31.0.101-preview.11.117</_AndroidRuntimePackVersion>
</PropertyGroup>
<PropertyGroup>
<_AndroidRuntimePackId Condition=" '%24(_AndroidRuntimePackId)' == '' ">$(AndroidLatestStableApiLevel)</_AndroidRuntimePackId>
<_AndroidRuntimePackVersion Condition=" '%24(_AndroidRuntimePackVersion)' == '' ">**FromWorkload**</_AndroidRuntimePackVersion>
</PropertyGroup>
<ItemGroup>
<KnownFrameworkReference
Include="Microsoft.Android"
TargetFramework="$(_AndroidNETAppTargetFramework)"
RuntimeFrameworkName="Microsoft.Android"
DefaultRuntimeFrameworkVersion="**FromWorkload**"
LatestRuntimeFrameworkVersion="**FromWorkload**"
LatestRuntimeFrameworkVersion="%24(_AndroidRuntimePackVersion)"
TargetingPackName="Microsoft.Android.Ref.%24(_AndroidRuntimePackId)"
TargetingPackVersion="**FromWorkload**"
TargetingPackVersion="%24(_AndroidRuntimePackVersion)"
RuntimePackNamePatterns="Microsoft.Android.Runtime.%24(_AndroidRuntimePackId).**RID**"
RuntimePackRuntimeIdentifiers="@(_AndroidNETAppRuntimePackRids, '%3B')"
Profile="Android"
Expand Down
1 change: 1 addition & 0 deletions build-tools/scripts/XABuildConfig.cs.in
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ namespace Xamarin.Android.Tools
public const string NDKRelease = "@NDK_RELEASE@";
public const int NDKMinimumApiAvailable = @NDK_MINIMUM_API_AVAILABLE@;
public const int AndroidLatestStableApiLevel = @ANDROID_LATEST_STABLE_API_LEVEL@;
public const int AndroidDefaultTargetDotnetApiLevel = @ANDROID_DEFAULT_TARGET_DOTNET_API_LEVEL@;
public static readonly Version NDKVersion = new Version (@NDK_VERSION_MAJOR@, @NDK_VERSION_MINOR@, @NDK_VERSION_MICRO@);

public static readonly Dictionary <string, int> ArchAPILevels = new Dictionary <string, int> (StringComparer.Ordinal) {
Expand Down
Loading