Add APK test job to azure-pipelines.yaml #3042
Merged
Commits on Jun 18, 2019
-
[RunInstrumentationTests] Error on test failure
The instrumentatione exit code value of `-1` indicates success, and any other values returned here should be reported as errors. A non `-1` value indicates that crash or test failure occurred.
-
[Adb] Don't always report stderr as a warning
Adb invocations are prone to returning non zero exit codes even when they are successful. This can make our msbuild output a bit noisy[0] when they are invoked using the Adb.cs task. The new optional parameter `WriteOutputAsMessage` can now be provided to the adb task, so that we don't always write messages coming from stderr as msbuild warnings. This is used in our targets which wait for emulator startup to make it easier to detect real failures in test instrumentation. [0] ``` /Users/peter/source/pj/xamarin-android/build-tools/scripts/TestApks.targets(68,5): warning : adb D 04-29 15:10:48 3527 42649 adb_trace.cpp:192] Android Debug Bridge version 1.0.40 [/Users/peter/source/pj/xamarin-android/tests/locales/Xamarin.Android.Locale-Tests/Xamarin.Android.Locale-Tests.csproj] /Users/peter/source/pj/xamarin-android/build-tools/scripts/TestApks.targets(68,5): warning : adb D 04-29 15:10:48 3527 42649 adb_trace.cpp:192] Version 28.0.2-5303910 [/Users/peter/source/pj/xamarin-android/tests/locales/Xamarin.Android.Locale-Tests/Xamarin.Android.Locale-Tests.csproj] /Users/peter/source/pj/xamarin-android/build-tools/scripts/TestApks.targets(68,5): warning : adb D 04-29 15:10:48 3527 42649 adb_trace.cpp:192] Installed as /Users/peter/android-toolchain/sdk/platform-tools/adb [/Users/peter/source/pj/xamarin-android/tests/locales/Xamarin.Android.Locale-Tests/Xamarin.Android.Locale-Tests.csproj] /Users/peter/source/pj/xamarin-android/build-tools/scripts/TestApks.targets(68,5): warning : adb D 04-29 15:10:48 3527 42649 adb_trace.cpp:192] [/Users/peter/source/pj/xamarin-android/tests/locales/Xamarin.Android.Locale-Tests/Xamarin.Android.Locale-Tests.csproj] /Users/peter/source/pj/xamarin-android/build-tools/scripts/TestApks.targets(68,5): warning : adb D 04-29 15:10:48 3527 42649 adb_client.cpp:287] adb_query: host-serial:emulator-5570:features [/Users/peter/source/pj/xamarin-android/tests/locales/Xamarin.Android.Locale-Tests/Xamarin.Android.Locale-Tests.csproj] /Users/peter/source/pj/xamarin-android/build-tools/scripts/TestApks.targets(68,5): warning : adb D 04-29 15:10:48 3527 42649 adb_client.cpp:137] _adb_connect: host:version [/Users/peter/source/pj/xamarin-android/tests/locales/Xamarin.Android.Locale-Tests/Xamarin.Android.Locale-Tests.csproj] /Users/peter/source/pj/xamarin-android/build-tools/scripts/TestApks.targets(68,5): warning : adb D 04-29 15:10:48 3527 42649 adb_io.cpp:107] writex: fd=3 len=16 30303063686f73743a76657273696f6e 000chost:version [/Users/peter/source/pj/xamarin-android/tests/locales/Xamarin.Android.Locale-Tests/Xamarin.Android.Locale-Tests.csproj] /Users/peter/source/pj/xamarin-android/build-tools/scripts/TestApks.targets(68,5): warning : adb D 04-29 15:10:48 3527 42649 adb_io.cpp:81] readx: fd=3 wanted=4 [/Users/peter/source/pj/xamarin-android/tests/locales/Xamarin.Android.Locale-Tests/Xamarin.Android.Locale-Tests.csproj] /Users/peter/source/pj/xamarin-android/build-tools/scripts/TestApks.targets(68,5): warning : adb D 04-29 15:10:48 3527 42649 adb_io.cpp:97] readx: fd=3 wanted=4 got=4 4f4b4159 OKAY [/Users/peter/source/pj/xamarin-android/tests/locales/Xamarin.Android.Locale-Tests/Xamarin.Android.Locale-Tests.csproj] /Users/peter/source/pj/xamarin-android/build-tools/scripts/TestApks.targets(68,5): warning : adb D 04-29 15:10:48 3527 42649 adb_client.cpp:165] _adb_connect: return fd 3 [/Users/peter/source/pj/xamarin-android/tests/locales/Xamarin.Android.Locale-Tests/Xamarin.Android.Locale-Tests.csproj] /Users/peter/source/pj/xamarin-android/build-tools/scripts/TestApks.targets(68,5): warning : adb D 04-29 15:10:48 3527 42649 adb_client.cpp:197] adb_connect: service host-serial:emulator-5570:features [/Users/peter/source/pj/xamarin-android/tests/locales/Xamarin.Android.Locale-Tests/Xamarin.Android.Locale-Tests.csproj] /Users/peter/source/pj/xamarin-android/build-tools/scripts/TestApks.targets(68,5): warning : adb D 04-29 15:10:48 3527 42649 adb_io.cpp:81] readx: fd=3 wanted=4 [/Users/peter/source/pj/xamarin-android/tests/locales/Xamarin.Android.Locale-Tests/Xamarin.Android.Locale-Tests.csproj] ```
-
-
[tests] Error if logcat timing data can't be parsed
We'll now error out when parsing logcat for timing data if the running application died, or if we fail to detect application startup.
-
-
[build] Use xaprepare to provision Android tools
[build] Test failures should not produce a publish error MSBuild will error if any test failures occur during our APK tests, and producing an extra error during the publish step only adds "noise" to the overall build health.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.