Skip to content

Commit

Permalink
[Xharness] Fix all xml parsing issues and ensure that the xml can be …
Browse files Browse the repository at this point in the history 
…consumed by VSTS. (#7768)

We ping the tcp listener to know that we have a tcp connection, that is
written in the xml logs, which means that parsing will not work. Ignore
the ping, parse xml, and make sure that the xml that will be consume by
vsts is valid.

This PR also fixes xamarin/maccore#827 which does not longer happen.
  • Loading branch information
@mandel-macaque
mandel-macaque authored Feb 4, 2020
1 parent ed16e04 commit 0144c3b
Show file tree
Hide file tree
Showing 9 changed files with 574 additions and 101 deletions.
11 changes: 8 additions & 3 deletions tests/bcl-test/BCLTests-mac.csproj.in
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,16 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Mono.Options" Version="5.3.0.1" />
<PackageReference Include="System.Buffers" Version="4.5.0" />
<PackageReference Include="System.Memory" Version="4.5.2" />
<PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="4.5.2" />
<PackageReference Include="System.Buffers" Version="4.4.0" />
<PackageReference Include="System.Memory" Version="4.5.1" />
<PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="4.5.1" />
<PackageReference Include="xunit" Version="2.4.0" />
<PackageReference Include="xunit.analyzers" Version="0.10.0" />
<PackageReference Include="xunit.extensibility.core" Version="2.4.0" />
<PackageReference Include="xunit.extensibility.execution" Version="2.4.0" />
<PackageReference Include="xunit.runner.utility" Version="2.4.0" />


</ItemGroup>
<ItemGroup>
<Reference Include="System">
Expand Down Expand Up @@ -99,6 +101,9 @@
<Compile Include="templates\common\TestRunner.NUnit\ClassOrNamespaceFilter.cs">
<Link>TestRunner.NUnit\ClassOrNamespaceFilter.cs</Link>
</Compile>
<Compile Include="templates\common\TestRunner.NUnit\XmlOutputWriter.cs">
<Link>TestRunner.NUnit\XmlOutputWriter.cs</Link>
</Compile>
<Compile Include="templates\common\TestRunner.NUnit\TestMethodFilter.cs">
<Link>TestRunner.NUnit\TestMethodFilter.cs</Link>
</Compile>
Expand Down
19 changes: 11 additions & 8 deletions tests/bcl-test/BCLTests-tv.csproj.in
View file
Original file line number Diff line number Diff line change
Expand Up @@ -122,15 +122,15 @@
<MtouchI18n>cjk,mideast,other,rare,west</MtouchI18n>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Mono.Options" Version="5.3.0.1" />
<PackageReference Include="System.Buffers" Version="4.4.0" />
<PackageReference Include="System.Memory" Version="4.5.1" />
<PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="4.5.1" />
<PackageReference Include="xunit" Version="2.4.0" />
<PackageReference Include="Mono.Options" Version="6.6.0.161" />
<PackageReference Include="System.Buffers" Version="4.5.0" />
<PackageReference Include="System.Memory" Version="4.5.3" />
<PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="4.7.0" />
<PackageReference Include="xunit" Version="2.4.1" />
<PackageReference Include="xunit.analyzers" Version="0.10.0" />
<PackageReference Include="xunit.extensibility.core" Version="2.4.0" />
<PackageReference Include="xunit.extensibility.execution" Version="2.4.0" />
<PackageReference Include="xunit.runner.utility" Version="2.4.0" />
<PackageReference Include="xunit.extensibility.core" Version="2.4.1" />
<PackageReference Include="xunit.extensibility.execution" Version="2.4.1" />
<PackageReference Include="xunit.runner.utility" Version="2.4.1" />
</ItemGroup>
<ItemGroup>
<Reference Include="System" />
Expand Down Expand Up @@ -176,6 +176,9 @@
<Compile Include="templates\common\TestRunner.NUnit\TestMethodFilter.cs">
<Link>TestRunner.NUnit\TestMethodFilter.cs</Link>
</Compile>
<Compile Include="templates\common\TestRunner.NUnit\XmlOutputWriter.cs">
<Link>TestRunner.NUnit\XmlOutputWriter.cs</Link>
</Compile>
<Compile Include="templates\common\TestRunner.Core\Extensions.Bool.cs">
<Link>TestRunner.Core\Extensions.Bool.cs</Link>
</Compile>
Expand Down
19 changes: 11 additions & 8 deletions tests/bcl-test/BCLTests-watchos-extension.csproj.in
View file
Original file line number Diff line number Diff line change
Expand Up @@ -128,15 +128,15 @@
<MtouchLink>SdkOnly</MtouchLink>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Mono.Options" Version="5.3.0.1" />
<PackageReference Include="System.Buffers" Version="4.4.0" />
<PackageReference Include="System.Memory" Version="4.5.1" />
<PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="4.5.1" />
<PackageReference Include="xunit" Version="2.4.0" />
<PackageReference Include="Mono.Options" Version="6.6.0.161" />
<PackageReference Include="System.Buffers" Version="4.5.0" />
<PackageReference Include="System.Memory" Version="4.5.3" />
<PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="4.7.0" />
<PackageReference Include="xunit" Version="2.4.1" />
<PackageReference Include="xunit.analyzers" Version="0.10.0" />
<PackageReference Include="xunit.extensibility.core" Version="2.4.0" />
<PackageReference Include="xunit.extensibility.execution" Version="2.4.0" />
<PackageReference Include="xunit.runner.utility" Version="2.4.0" />
<PackageReference Include="xunit.extensibility.core" Version="2.4.1" />
<PackageReference Include="xunit.extensibility.execution" Version="2.4.1" />
<PackageReference Include="xunit.runner.utility" Version="2.4.1" />
</ItemGroup>
<ItemGroup>
<Reference Include="System" />
Expand Down Expand Up @@ -206,6 +206,9 @@
<Compile Include="templates\common\TestRunner.NUnit\TestMethodFilter.cs">
<Link>TestRunner.NUnit\TestMethodFilter.cs</Link>
</Compile>
<Compile Include="templates\common\TestRunner.NUnit\XmlOutputWriter.cs">
<Link>TestRunner.NUnit\XmlOutputWriter.cs</Link>
</Compile>
<Compile Include="templates\common\TestRunner.xUnit\XUnitFilter.cs">
<Link>TestRunner.xUnit\XUnitFilter.cs</Link>
</Compile>
Expand Down
19 changes: 11 additions & 8 deletions tests/bcl-test/BCLTests.csproj.in
View file
Original file line number Diff line number Diff line change
Expand Up @@ -135,15 +135,15 @@
<MtouchI18n>cjk,mideast,other,rare,west</MtouchI18n>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Mono.Options" Version="5.3.0.1" />
<PackageReference Include="System.Buffers" Version="4.4.0" />
<PackageReference Include="System.Memory" Version="4.5.1" />
<PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="4.5.1" />
<PackageReference Include="xunit" Version="2.4.0" />
<PackageReference Include="Mono.Options" Version="6.6.0.161" />
<PackageReference Include="System.Buffers" Version="4.5.0" />
<PackageReference Include="System.Memory" Version="4.5.3" />
<PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="4.7.0" />
<PackageReference Include="xunit" Version="2.4.1" />
<PackageReference Include="xunit.analyzers" Version="0.10.0" />
<PackageReference Include="xunit.extensibility.core" Version="2.4.0" />
<PackageReference Include="xunit.extensibility.execution" Version="2.4.0" />
<PackageReference Include="xunit.runner.utility" Version="2.4.0" />
<PackageReference Include="xunit.extensibility.core" Version="2.4.1" />
<PackageReference Include="xunit.extensibility.execution" Version="2.4.1" />
<PackageReference Include="xunit.runner.utility" Version="2.4.1" />
</ItemGroup>
<ItemGroup>
<Reference Include="System" />
Expand Down Expand Up @@ -189,6 +189,9 @@
<Compile Include="templates\common\TestRunner.NUnit\TestMethodFilter.cs">
<Link>TestRunner.NUnit\TestMethodFilter.cs</Link>
</Compile>
<Compile Include="templates\common\TestRunner.NUnit\XmlOutputWriter.cs">
<Link>TestRunner.NUnit\XmlOutputWriter.cs</Link>
</Compile>
<Compile Include="templates\common\TestRunner.Core\Extensions.Bool.cs">
<Link>TestRunner.Core\Extensions.Bool.cs</Link>
</Compile>
Expand Down
2 changes: 0 additions & 2 deletions tests/bcl-test/templates/common/TestRunner.NUnit/NUnitTestRunner.cs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,8 @@

using Foundation;

using NUnitLite.Runner;
using NUnit.Framework.Api;
using NUnit.Framework.Internal;
using NUnit.Framework.Internal.WorkItems;
using NUnit.Framework.Internal.Filters;

using NUnitTest = NUnit.Framework.Internal.Test;
Expand Down
Loading

4 comments on commit 0144c3b

@xamarin-release-manager
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚧 Experimental DDFun pipeline

🔥 Device tests completed (Failed) on iOS-DDFun on Azure DevOps(iOS-DDFun) 🔥

Test results

14 tests failed, 128 tests passed.

Failed tests

  • monotouch-test/iOS Unified 64-bits - device/Debug: Failed
  • monotouch-test/iOS Unified 64-bits - device/AssemblyBuildTarget: dylib (debug): Failed
  • monotouch-test/iOS Unified 64-bits - device/AssemblyBuildTarget: SDK framework (debug): Failed
  • monotouch-test/iOS Unified 64-bits - device/AssemblyBuildTarget: dylib (debug, profiling): Failed
  • monotouch-test/iOS Unified 64-bits - device/AssemblyBuildTarget: SDK framework (debug, profiling): Failed
  • monotouch-test/iOS Unified 64-bits - device/Release: Failed
  • monotouch-test/iOS Unified 64-bits - device/AssemblyBuildTarget: SDK framework (release): Failed
  • monotouch-test/iOS Unified 64-bits - device/Debug (dynamic registrar): Failed
  • monotouch-test/iOS Unified 64-bits - device/Release (all optimizations): Failed
  • monotouch-test/iOS Unified 64-bits - device/Debug (all optimizations): Failed
  • monotouch-test/iOS Unified 64-bits - device/Debug: SGenConc: Failed
  • monotouch-test/iOS Unified 64-bits - device/Debug (interpreter): Failed
  • monotouch-test/iOS Unified 64-bits - device/Debug (interpreter -mscorlib): Failed
  • monotouch-test/iOS Unified 64-bits - device/Release (interpreter -mscorlib): Failed

@xamarin-release-manager
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Build was (probably) aborted

🔥 Jenkins job (on internal Jenkins) failed in stage(s) 'Test run, Test run' 🔥

✅ Build succeeded
✅ Packages:

✅ API Diff (from stable)
✅ API Diff (from PR only) (no change)
✅ Generator Diff (no change)
🔥 Test run failed 🔥

Test results

2 tests failed, 177 tests passed.

Failed tests

  • monotouch-test/watchOS 32-bits - simulator/Debug: Crashed
  • MTouch tests/NUnit: Failed (Execution failed with exit code 7)

@xamarin-release-manager
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥 Device tests completed (Failed) on iOS on Azure DevOps(iOS): Html Report 🔥

Test results

2 tests failed, 140 tests passed.

Failed tests

  • link sdk/iOS Unified 64-bits - device/AssemblyBuildTarget: SDK framework (debug): Failed
  • [xUnit] Mono SystemCoreXunit Part 2/iOS Unified 64-bits - device/AssemblyBuildTarget: SDK framework (release): Failed

@xamarin-release-manager
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥 Device tests completed (Failed) on TvOS on Azure DevOps(TvOS): Html Report 🔥

Test results

# Test run in progress: Building: 1, BuildQueued: 4, Running: 1, Succeeded: 133, Ignored: 1070, TimedOut: 3

Failed tests

  • [xUnit] Mono SystemCoreXunit Part 1/tvOS - device/Debug: TimedOut
  • [xUnit] Mono BCL tests group 5/tvOS - device/AssemblyBuildTarget: SDK framework (release): TimedOut
  • [xUnit] Mono SystemCoreXunit Part 1/tvOS - device/AssemblyBuildTarget: SDK framework (debug): TimedOut

Please sign in to comment.