From 3ae3aa131ed2e513694787b5aee07a7e6f044f55 Mon Sep 17 00:00:00 2001 From: Konrad Jamrozik Date: Thu, 22 Dec 2022 17:54:18 -0800 Subject: [PATCH] clarify Runtime matrix generation customization --- eng/common/scripts/job-matrix/README.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/eng/common/scripts/job-matrix/README.md b/eng/common/scripts/job-matrix/README.md index ba5f4768ba6..00e0ba936a3 100644 --- a/eng/common/scripts/job-matrix/README.md +++ b/eng/common/scripts/job-matrix/README.md @@ -3,6 +3,7 @@ * [Azure Pipelines Matrix Generator](#azure-pipelines-matrix-generator) * [How to use matrix generator from your pipeline](#how-to-use-matrix-generator-from-your-pipeline) * [Matrix generator usage example](#matrix-generator-usage-example) + * [Runtime matrix generation customization](#runtime-matrix-generation-customization) * [Matrix config file syntax](#matrix-config-file-syntax) * [Fields](#fields) * [matrix](#matrix) @@ -90,12 +91,15 @@ jobs: To see an example of a complete pipeline definition with a job that runs your job using matrix generator, refer to [`/eng/common/scripts/job-matrix/samples/matrix-test.yml`](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/scripts/job-matrix/samples/matrix-test.yml). -### A note regarding PreGenerationSteps +### Runtime matrix generation customization -The generation template laid out above runs as its own job. A limitation of this method is that it disallows any runtime matrix customization due to the fact that an individual job clones the targeted build SHA. +The "matrix generator-enabled" job laid out above runs as its own job. A limitation of this approach is that it disallows any runtime matrix customization due to the fact that an individual job clones the targeted build SHA. That is, the matrix to generate would be determined based only on the content of the used matrix json configs. -To address this limitation, we introduce the [stepList](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#parameter-data-types) `PreGenerationSteps`. -It allows users to update matrix config json however they like prior to actually invoking the matrix generation. Injected steps are run after the repository checkout, but before any matrix generation is invoked. +To address this limitation, we introduce the [stepList](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#parameter-data-types) `PreGenerationSteps` as well as `MatrixFilters` and `MatrixReplace`. + +`PreGenerationSteps` allows users to update matrix config json however they like prior to actually invoking the matrix generation. Injected steps are run after the repository checkout, but before any matrix generation is invoked. + +`MatrixFilters` and `MatrixReplace` allow runtime adjustment of the matrix generation process as can be seen in the source of [GenerateMatrix](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/scripts/job-matrix/job-matrix-functions.ps1#L94-L95). ## Matrix config file syntax