Skip to content

Commit

Permalink
Separate Helix and XHarness testing (#8255)
Browse files Browse the repository at this point in the history
* Separate Helix and XHarness testing

There are frequently intermittent failures in the XHarness testing that need to be retried due to timeouts or xharness flakiness. Because the XHarness tests are in the same job as the regular helix tests, we have rerun all of the testing just to rerun the xharness tests. This wastes resources and is frustrating for devs.

This change splits out the xharness tests into their own jobs so that when xharness reruns need to happen, we only have to rerun the xharness legs.

* Fix publishing logs

* And there

* Fix yaml syntax, separate even further
  • Loading branch information
michellemcdaniel authored Dec 10, 2021
1 parent 04eac97 commit 18adc5b
Showing 1 changed file with 73 additions and 2 deletions.
75 changes: 73 additions & 2 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -102,9 +102,9 @@ stages:
artifacts:
publish:
artifacts:
name: Artifacts_Test_$(Agent.OS)_$(_BuildConfig)
name: Artifacts_Test_$(Agent.OS)_$(_BuildConfig)_$(_Testing)
logs:
name: Logs_Test_$(Agent.OS)_$(_BuildConfig)
name: Logs_Test_$(Agent.OS)_$(_BuildConfig)_$(_Testing)
download: true
workspace:
clean: all
Expand All @@ -119,6 +119,8 @@ stages:
_BuildConfig: Release
Build_Debug:
_BuildConfig: Debug
variables:
- _Testing: Helix
preSteps:
- checkout: self
clean: true
Expand Down Expand Up @@ -148,6 +150,22 @@ stages:
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
HelixAccessToken: ''
- job: Windows_NT_XHarness
timeoutInMinutes: 90
pool:
vmimage: windows-latest
strategy:
matrix:
Build_Release:
_BuildConfig: Release
Build_Debug:
_BuildConfig: Debug
variables:
- _Testing: Xharness
preSteps:
- checkout: self
clean: true
steps:
- powershell: eng\common\build.ps1
-configuration $(_BuildConfig)
-prepareMachine
Expand All @@ -172,6 +190,8 @@ stages:
_BuildConfig: Debug
Build_Release:
_BuildConfig: Release
variables:
- _Testing: Helix
preSteps:
- checkout: self
clean: true
Expand All @@ -189,6 +209,23 @@ stages:
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
HelixAccessToken: ''
- job: Linux_XHarness_Apple_Simulator
timeoutInMinutes: 90
container: LinuxContainer
pool:
vmimage: ubuntu-latest
strategy:
matrix:
Build_Debug:
_BuildConfig: Debug
Build_Release:
_BuildConfig: Release
variables:
- _Testing: Xharness_Apple_Simulator
preSteps:
- checkout: self
clean: true
steps:
- script: eng/common/build.sh
-configuration $(_BuildConfig)
-prepareMachine
Expand All @@ -202,6 +239,23 @@ stages:
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
HelixAccessToken: ''
- job: Linux_XHarness_Apple_Device
timeoutInMinutes: 90
container: LinuxContainer
pool:
vmimage: ubuntu-latest
strategy:
matrix:
Build_Debug:
_BuildConfig: Debug
Build_Release:
_BuildConfig: Release
variables:
- _Testing: Xharness_Apple_Device
preSteps:
- checkout: self
clean: true
steps:
- script: eng/common/build.sh
-configuration $(_BuildConfig)
-prepareMachine
Expand All @@ -215,6 +269,23 @@ stages:
env:
SYSTEM_ACCESSTOKEN: $(System.AccessToken)
HelixAccessToken: ''
- job: Linux_XHarness
timeoutInMinutes: 90
container: LinuxContainer
pool:
vmimage: ubuntu-latest
strategy:
matrix:
Build_Debug:
_BuildConfig: Debug
Build_Release:
_BuildConfig: Release
variables:
- _Testing: Xharness
preSteps:
- checkout: self
clean: true
steps:
- script: eng/common/build.sh
-configuration $(_BuildConfig)
-prepareMachine
Expand Down

0 comments on commit 18adc5b

Please sign in to comment.