Linux-bionic (Android libc w/o Android interop) initial implementation #66147

directhex · 2022-03-03T16:48:42Z

Implement support for "Linux Bionic", a RID which builds a Linux runtime against an Android libc but without Android OS functions like JNI, Dalvik, Instrumentation, etc. This configuration has been requested by a team working on an IoT scenario.

The new RID is linux-bionic-arm64/linux-bionic-x64.
The build reuses the existing Android build dockerfiles and Helix queues
Since we don't have JNI, we don't have Android Crypto - instead, use an Android build of OpenSSL. As an errata, it is incumbent upon the end user to supply this in their OS environment (we download one ad-hoc for testing purposes, which was compiled by Google)
Some tests cannot be completed due to SELinux policies affecting the shell user executing tests via adb shell in the world-writable /data/local/tmp directory. We expect those tests would pass if they pass on Android, in a real-world IoT scenario with a custom OS (rather than recycling Google's OS images intended for phones)

Closes: #66027

Closes: dotnet#66027

dotnet-issue-labeler · 2022-03-03T16:48:46Z

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

ghost · 2022-03-03T18:24:53Z

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

I expect I broke something along the way, so let's see just how badly.

Closes: #66027

Author:	directhex
Assignees:	directhex
Labels:	`area-Infrastructure-mono`
Milestone:	-

…into joshield/linux-bionic

src/libraries/Microsoft.NETCore.Platforms/src/runtimeGroups.props

…into joshield/linux-bionic

directhex · 2022-03-07T18:40:21Z

Urgh, horrible merge failure means I spent hours cleaning up unneeded changes, and they reappeared anyway

directhex · 2022-03-07T18:57:23Z

New regression: SSL doesn't work because an Android System.Net.Security is being used instead of a Linux one, which causes it to forcibly load Android SSL instead of OpenSSL.

steveisok · 2022-03-07T19:54:49Z

New regression: SSL doesn't work because an Android System.Net.Security is being used instead of a Linux one, which causes it to forcibly load Android SSL instead of OpenSSL.

I don't think parts of System.Net.Security are going to work on Android anyway. At least not in .NET 6.

directhex · 2022-03-07T19:56:26Z

New regression: SSL doesn't work because an Android System.Net.Security is being used instead of a Linux one, which causes it to forcibly load Android SSL instead of OpenSSL.

I don't think System.Net.Security is going to work on Android anyway. At least not in .NET 6.

That's kinda the point. It worked yesterday when the build system assumed Bionic was Linux, and it's broken today now the build system thinks Bionic is Android, and I'm trying to figure out how to massage that in the build system to behave better (msbuild log viewer shows none of my properties in System.Net.Security.csproj are getting resolved, something odd is going on)

steveisok · 2022-03-07T20:00:09Z

Tweak this and you should be ok: https://github.com/directhex/runtime/blob/78111b553b2546356130a1d7ed33c944f3a94e46/src/libraries/System.Net.Security/src/System.Net.Security.csproj#L14

directhex · 2022-03-07T20:12:15Z

Tweak this and you should be ok: https://github.com/directhex/runtime/blob/78111b553b2546356130a1d7ed33c944f3a94e46/src/libraries/System.Net.Security/src/System.Net.Security.csproj#L14

You'd think! And yet

Hence "and something odd is going on"

directhex · 2022-03-07T20:32:24Z

Looks like the problem is with the structured log viewer, I see more via /v:diag

directhex · 2022-03-09T18:46:02Z

Okay, so, need help from someone who understands the libraries build a little better here (@ericstj?)

The build doesn't really support OS subgroups (i.e. linux-musl, linux-bionic), it's a gross hack where the system thinks it's actually the OSGroup OS (Linux for musl), some magic autodetection code in a few shell scripts tells it otherwise and causes changes in build behaviour, and the RID gets overridden post-hoc via /p:OutputRID. I've removed this assumption in a few places and parsed /p:RuntimeOS, because it's not possible to autodetect the difference between Android and linux-bionic based on anything in the rootfs.
We still need a TargetOS (--os). The linux-musl build defaults to linux; in our case there's a case to be made for both linux and android. And we kinda need to be both, in a few places - System.Net.Security and System.Security.Cryptography we need the Linux version NOT the Android version, whereas System.Console we want the Android version. Do we want to add a separate TFM for linux-bionic? Use RuntimeOS conditionals in the relevant csproj? What's the least painful way to do this?

directhex · 2022-05-11T19:52:19Z

Okay, there's a few small bits & pieces to clean up, but the broad strokes are there (e.g. the on-device build is green and the tests are published), so I'm gonna mark this ready for review. Hopefully we can nail down any lingering issues by the end of the week, and clean up any questionable conditions to everyone's satisfaction.

am11

LGTM. Thank you.

src/native/libs/CMakeLists.txt

akoeplinger

Great jobs, I left a couple comments but this is really close :)

Directory.Build.props

eng/Subsets.props

eng/native/naming.props

eng/pipelines/common/global-build-job.yml

akoeplinger · 2022-05-17T19:35:03Z

src/native/corehost/hostmisc/pal.h

@@ -83,6 +83,8 @@
 #define FALLBACK_HOST_RID _X("solaris")
 #elif defined(TARGET_LINUX_MUSL)
 #define FALLBACK_HOST_RID _X("linux-musl")
+#elif defined(TARGET_ANDROID)


it'd probably be better to introduce a TARGET_LINUX_BIONIC define instead.

akoeplinger · 2022-05-17T19:36:37Z

src/mono/mono.proj

      <_MonoRunInitCompiler>false</_MonoRunInitCompiler>
    </PropertyGroup>
-    <ItemGroup Condition="'$(TargetsAndroid)' == 'true'">
+    <ItemGroup Condition="'$(TargetsLinuxBionic)' == 'true'">
+      <_MonoCPPFLAGS Include="-DANDROID_FORCE_ICU_DATA_DIR" />


nit: call this FORCE_ANDROID_ICU_DATA_DIR to make it consistent with FORCE_ANDROID_OPENSSL (or change the other one, I don't have a preference)

probably better to move this to the src/native CMakeFiles anyway though.

src/native/libs/System.Security.Cryptography.Native/extra_libs.cmake

src/native/libs/System.Security.Cryptography.Native/opensslshim.c

src/native/libs/build-native.proj

Directory.Build.props

akoeplinger

Thanks, the rest of my comments can be addressed in followup-PRs

directhex added 2 commits March 3, 2022 11:34

Initial linux-bionic implementation.

03ed935

Closes: dotnet#66027

Add some CI

6043460

ghost assigned directhex Mar 3, 2022

Fix the build

8c5f357

danmoseley added the area-Infrastructure-mono label Mar 3, 2022

Merge remote-tracking branch 'origin/main' into joshield/linux-bionic

c3eb11e

steveisok requested review from steveisok and akoeplinger March 3, 2022 21:08

directhex added 2 commits March 4, 2022 15:21

Update compat

09c50b6

Merge branch 'joshield/linux-bionic' of github.com:directhex/runtime …

d23ae61

…into joshield/linux-bionic

marek-safar reviewed Mar 7, 2022

View reviewed changes

src/libraries/Microsoft.NETCore.Platforms/src/runtimeGroups.props Outdated Show resolved Hide resolved

runfoapp bot mentioned this pull request Mar 7, 2022

System.IO.Tests work item failing with SIGKILL #65791

Closed

directhex added 4 commits March 7, 2022 13:18

WIP to make this smaller & use -os android as a base

50f245a

Merge branch 'joshield/linux-bionic' of github.com:directhex/runtime …

416a094

…into joshield/linux-bionic

Merge remote-tracking branch 'origin/main' into joshield/linux-bionic

f87a238

Leftover crud

f91bc32

directhex added 2 commits March 7, 2022 13:48

More cleanup

eba386a

Refreshed runtime compat after inverting Android parenthood

78111b5

directhex added 2 commits March 9, 2022 10:47

Fix build to use -os Linux as a base, not -os Android

d41e9b8

Another stupid merge failure from days ago

cf290bf

directhex added 3 commits May 11, 2022 13:21

Bump xharness to fix failed tests plus all tests on Windows

5e3368d

Merge remote-tracking branch 'origin/main' into joshield/linux-bionic

7a118d1

Make WindowsShell usage more specific

df33943

directhex marked this pull request as ready for review May 11, 2022 19:52

directhex added 2 commits May 11, 2022 17:47

Add exclusions for failing tests. Let's get green.

5e60931

Fix typo

f45adbf

directhex requested a review from am11 May 12, 2022 14:55

steveisok approved these changes May 12, 2022

View reviewed changes

am11 approved these changes May 13, 2022

View reviewed changes

src/native/libs/CMakeLists.txt Show resolved Hide resolved

directhex added 3 commits May 16, 2022 08:28

Merge remote-tracking branch 'origin/main' into joshield/linux-bionic

7685f14

typo

fdece95

wat

b86c675

akoeplinger requested changes May 17, 2022

View reviewed changes

ghost added the needs-author-action An issue or pull request that requires more info or actions from the author. label May 17, 2022

Merge remote-tracking branch 'origin/main' into joshield/linux-bionic

5f93973

ghost removed the needs-author-action An issue or pull request that requires more info or actions from the author. label May 19, 2022

directhex mentioned this pull request May 19, 2022

Cannot easily generate bash RunTests scripts from Windows, and vice versa #69555

Open

directhex added 4 commits May 19, 2022 09:35

Various change suggestions from review by Alex

5499ed6

additional

f7d9bdd

Consistency w/ musl

b664de9

Consistency

ce2d7fc

akoeplinger reviewed May 19, 2022

View reviewed changes

Directory.Build.props Outdated Show resolved Hide resolved

directhex added 2 commits May 19, 2022 09:49

Redundant

9343257

sigh

9868647

akoeplinger approved these changes May 19, 2022

View reviewed changes

Maybe move a property

a316320

directhex merged commit b5f9c22 into dotnet:main May 19, 2022

MichalStrehovsky mentioned this pull request Jun 14, 2022

CoreCLR, CoreRT and Android #63384

Closed

ghost locked as resolved and limited conversation to collaborators Jun 19, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Linux-bionic (Android libc w/o Android interop) initial implementation #66147

Linux-bionic (Android libc w/o Android interop) initial implementation #66147

directhex commented Mar 3, 2022 •

edited

Loading

dotnet-issue-labeler bot commented Mar 3, 2022

ghost commented Mar 3, 2022

directhex commented Mar 7, 2022

directhex commented Mar 7, 2022

steveisok commented Mar 7, 2022 •

edited

Loading

directhex commented Mar 7, 2022

steveisok commented Mar 7, 2022

directhex commented Mar 7, 2022

directhex commented Mar 7, 2022

directhex commented Mar 9, 2022

directhex commented May 11, 2022

am11 left a comment

akoeplinger left a comment

akoeplinger May 17, 2022

akoeplinger May 17, 2022

akoeplinger May 17, 2022

akoeplinger left a comment

Linux-bionic (Android libc w/o Android interop) initial implementation #66147

Linux-bionic (Android libc w/o Android interop) initial implementation #66147

Conversation

directhex commented Mar 3, 2022 • edited Loading

dotnet-issue-labeler bot commented Mar 3, 2022

ghost commented Mar 3, 2022

directhex commented Mar 7, 2022

directhex commented Mar 7, 2022

steveisok commented Mar 7, 2022 • edited Loading

directhex commented Mar 7, 2022

steveisok commented Mar 7, 2022

directhex commented Mar 7, 2022

directhex commented Mar 7, 2022

directhex commented Mar 9, 2022

directhex commented May 11, 2022

am11 left a comment

Choose a reason for hiding this comment

akoeplinger left a comment

Choose a reason for hiding this comment

akoeplinger May 17, 2022

Choose a reason for hiding this comment

akoeplinger May 17, 2022

Choose a reason for hiding this comment

akoeplinger May 17, 2022

Choose a reason for hiding this comment

akoeplinger left a comment

Choose a reason for hiding this comment

directhex commented Mar 3, 2022 •

edited

Loading

steveisok commented Mar 7, 2022 •

edited

Loading