Skip to content

Commit

Permalink
clarify Runtime matrix generation customization
Browse files Browse the repository at this point in the history
  • Loading branch information
Konrad Jamrozik committed Dec 23, 2022
1 parent 9599c5a commit 3ae3aa1
Showing 1 changed file with 8 additions and 4 deletions.
12 changes: 8 additions & 4 deletions eng/common/scripts/job-matrix/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
* [Azure Pipelines Matrix Generator](#azure-pipelines-matrix-generator)
* [How to use matrix generator from your pipeline](#how-to-use-matrix-generator-from-your-pipeline)
* [Matrix generator usage example](#matrix-generator-usage-example)
* [Runtime matrix generation customization](#runtime-matrix-generation-customization)
* [Matrix config file syntax](#matrix-config-file-syntax)
* [Fields](#fields)
* [matrix](#matrix)
Expand Down Expand Up @@ -90,12 +91,15 @@ jobs:
To see an example of a complete pipeline definition with a job that runs your job using matrix generator, refer to
[`/eng/common/scripts/job-matrix/samples/matrix-test.yml`](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/scripts/job-matrix/samples/matrix-test.yml).

### A note regarding PreGenerationSteps
### Runtime matrix generation customization

The generation template laid out above runs as its own job. A limitation of this method is that it disallows any runtime matrix customization due to the fact that an individual job clones the targeted build SHA.
The "matrix generator-enabled" job laid out above runs as its own job. A limitation of this approach is that it disallows any runtime matrix customization due to the fact that an individual job clones the targeted build SHA. That is, the matrix to generate would be determined based only on the content of the used matrix json configs.

To address this limitation, we introduce the [stepList](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#parameter-data-types) `PreGenerationSteps`.
It allows users to update matrix config json however they like prior to actually invoking the matrix generation. Injected steps are run after the repository checkout, but before any matrix generation is invoked.
To address this limitation, we introduce the [stepList](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#parameter-data-types) `PreGenerationSteps` as well as `MatrixFilters` and `MatrixReplace`.

`PreGenerationSteps` allows users to update matrix config json however they like prior to actually invoking the matrix generation. Injected steps are run after the repository checkout, but before any matrix generation is invoked.

`MatrixFilters` and `MatrixReplace` allow runtime adjustment of the matrix generation process as can be seen in the source of [GenerateMatrix](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/scripts/job-matrix/job-matrix-functions.ps1#L94-L95).

## Matrix config file syntax

Expand Down

0 comments on commit 3ae3aa1

Please sign in to comment.