From fe603e2cb7b60d205b2782ccaa19635c521e9116 Mon Sep 17 00:00:00 2001 From: Konrad Jamrozik Date: Thu, 22 Dec 2022 21:48:31 -0800 Subject: [PATCH] ongoing --- eng/common/scripts/job-matrix/README.md | 91 ++++++++++++++----------- 1 file changed, 53 insertions(+), 38 deletions(-) diff --git a/eng/common/scripts/job-matrix/README.md b/eng/common/scripts/job-matrix/README.md index 8b780e87a77..d55fafef4d4 100644 --- a/eng/common/scripts/job-matrix/README.md +++ b/eng/common/scripts/job-matrix/README.md @@ -1,8 +1,9 @@ # Azure Pipelines Matrix Generator * [Azure Pipelines Matrix Generator](#azure-pipelines-matrix-generator) +* [How does the matrix generator work](#how-does-the-matrix-generator-work) * [How to use matrix generator from your pipeline](#how-to-use-matrix-generator-from-your-pipeline) - * [Matrix generator usage example](#matrix-generator-usage-example) + * [Matrix generator pipeline usage example](#matrix-generator-pipeline-usage-example) * [Runtime matrix generation customization](#runtime-matrix-generation-customization) * [Matrix config file syntax](#matrix-config-file-syntax) * [Fields](#fields) @@ -34,9 +35,18 @@ This implementation aims to address that, by replicating the cross-product matri and [excludes](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrixexclude) filters, but also adds some additional features like sparse matrix generation and programmable matrix filters. +## How does the matrix generator work + This matrix generator implementation works by generating a json value for [`jobs.job.strategy.matrix`](https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema/jobs-job-strategy?view=azure-pipelines#strategy-matrix-maxparallel) and passing it to the definition, which is possible because [matrix can accept a runtime expression containing a stringified json object](https://docs.microsoft.com/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml#multi-job-configuration) (see the code sample at the bottom of the linked section). +You can use the matrix generator in two ways: [from the pipeline](#how-to-use-matrix-generator-from-your-pipeline), +or by calling [`Create-JobMatrix.ps1`](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/scripts/job-matrix/Create-JobMatrix.ps1) script directly and then passing the generated matrix json as argument to `jobs.job.strategy.matrix`. + +The pipeline usage is recommended. + +If you call the generator from the pipeline, you will rely on the [archetype](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/pipelines/templates/jobs/archetype-sdk-tests-generate.yml), which ends up calling `Create-JobMatrix.ps1` behind the scenes. + ## How to use matrix generator from your pipeline Assume you have a job defined in azure pipelines yaml file. You want to run @@ -48,7 +58,7 @@ based on one or more matrix json configs referenced in its `MatrixConfigs` param That job will use as template the definition [`archetype-sdk-tests-generate.yml`](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/pipelines/templates/jobs/archetype-sdk-tests-generate.yml). -### Matrix generator usage example +### Matrix generator pipeline usage example Here is an example. Let's assume your job definition has following path: @@ -163,13 +173,13 @@ Example: "matrix": { "operatingSystem": [ "windows-2022", - "ubuntu-18.04", + "ubuntu-22.04", "macos-11" ], "framework": [ "net461", "netcoreapp2.1", - "net50" + "net6.0" ], "additionalTestArguments": [ "", @@ -458,9 +468,9 @@ In the matrix job output that azure pipelines consumes, the format is a map of m "framework": "net461", "operatingSystem": "macos-11" }, - "net50_ubuntu1804": { - "framework": "net50", - "operatingSystem": "ubuntu-18.04" + "net60_ubuntu2204": { + "framework": "net6.0", + "operatingSystem": "ubuntu-22.04" }, "netcoreapp21_windows2022": { "framework": "netcoreapp2.1", @@ -507,16 +517,19 @@ named `ExcludedKey`, a `framework` parameter with value either `461` or `5.0`, a -Filters @("ExcludedKey=^$", "framework=(461|5\.0)", "SupportedClouds=^$|.*Public.*") ``` +Note that `Create-JobMatrix.ps1` is [called internally by the archetype](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/pipelines/templates/jobs/archetype-sdk-tests-generate.yml#L72) and you are never expected to call it directly. +Instead, use use the generator from the pipelines, as explained in [Matrix generator pipeline usage example](#matrix-generator-pipeline-usage-example). + ### Replace/Modify/Append Values Replacements for values can be passed to the matrix as an array of strings, in `MatrixReplace` parameter, each matching the format of `=/`. -The replace argument will find any combinations where the key fully matches the key regex and the value fully matches the value regex, and replace the value with -the replacement specified. +The replace argument will find any combinations where the key fully matches the key regex and the value fully matches +the value regex, and replace the value with the replacement specified. NOTE: * The replacement value supports regex capture groups, enabling substring transformations, e.g. `Foo=(.*)-replaceMe/$1-replaced`. See the below examples for usage. -* For each key/value, the first replacement provided that matches will be the only one applied. +* For each key/value pair, the first replacement provided that matches will be the only one applied. * If `=` or `/` characters need to be part of the regex or replacement, escape them with `\`. For example, given a matrix config like below: @@ -525,7 +538,7 @@ For example, given a matrix config like below: { "matrix": { "Agent": { - "ubuntu-1804": { "OSVmImage": "MMSUbuntu18.04", "Pool": "azsdk-pool-mms-ubuntu-1804-general" } + "ubuntu-2204": { "OSVmImage": "MMSUbuntu22.04", "Pool": "azsdk-pool-mms-ubuntu-2204-general" } }, "JavaTestVersion": [ "1.8", "1.11" ] } @@ -535,24 +548,24 @@ For example, given a matrix config like below: The normal matrix output (without replacements), looks like: -``` powershell +``` json $ ./Create-JobMatrix.ps1 -ConfigPath -Selection all { - "ubuntu1804_18": { - "OSVmImage": "MMSUbuntu18.04", - "Pool": "azsdk-pool-mms-ubuntu-1804-general", + "ubuntu2204_18": { + "OSVmImage": "MMSUbuntu22.04", + "Pool": "azsdk-pool-mms-ubuntu-2204-general", "JavaTestVersion": "1.8" }, - "ubuntu1804_111": { + "ubuntu2204_111": { "OSVmImage": "MMSUbuntu18.04", - "Pool": "azsdk-pool-mms-ubuntu-1804-general", + "Pool": "azsdk-pool-mms-ubuntu-2204-general", "JavaTestVersion": "1.11" } } ``` -Passing in multiple replacements, the output will look like below. Note that replacing key/values that appear nested within a grouping -will not affect that segment of the job name, since the job takes the grouping name (in this case "ubuntu1804"). +Passing in multiple replacements, the output will look like below. Note that replacing key/value pairs that appear +nested within a grouping will not affect that segment of the job name, since the job takes the grouping name (in this case `ubuntu2204`). The below example includes samples of regex grouping references, and wildcard key/value regexes: @@ -560,23 +573,24 @@ The below example includes samples of regex grouping references, and wildcard ke $ $replacements = @('.*Version=1.11/2.0', 'Pool=(.*ubuntu.*)-general/$1-custom') $ ../Create-JobMatrix.ps1 -ConfigPath ./test.Json -Selection all -Replace $replacements { - "ubuntu1804_18": { - "OSVmImage": "MMSUbuntu18.04", - "Pool": "azsdk-pool-mms-ubuntu-1804-custom", + "ubuntu2204_18": { + "OSVmImage": "MMSUbuntu22.04", + "Pool": "azsdk-pool-mms-ubuntu-2204-custom", "JavaTestVersion": "1.8" }, "ubuntu1804_20": { - "OSVmImage": "MMSUbuntu18.04", - "Pool": "azsdk-pool-mms-ubuntu-1804-custom", + "OSVmImage": "MMSUbuntu22.04", + "Pool": "azsdk-pool-mms-ubuntu-2204-custom", "JavaTestVersion": "2.0" } } ``` -#### NonSparseParameters +### NonSparseParameters -Sometimes it may be necessary to generate a sparse matrix, but keep the full combination of a few parameters. The -NonSparseParameters argument allows for more fine-grained control of matrix generation. For example: +Sometimes it may be necessary to generate a sparse matrix, but keep the full combination of a few parameters. +The `MatrixConfigs` `NonSparseParameters` parameter allows for more fine-grained control of matrix generation. +For example: ``` powershell ./Create-JobMatrix.ps1 ` @@ -592,7 +606,7 @@ Given a matrix like below with `JavaTestVersion` marked as a non-sparse paramete "matrix": { "Agent": { "windows-2022": { "OSVmImage": "MMS2022", "Pool": "azsdk-pool-mms-win-2022-general" }, - "ubuntu-1804": { "OSVmImage": "MMSUbuntu18.04", "Pool": "azsdk-pool-mms-ubuntu-1804-general" }, + "ubuntu-2204": { "OSVmImage": "MMSUbuntu22.04", "Pool": "azsdk-pool-mms-ubuntu-2204-general" }, "macos-11": { "OSVmImage": "macos-11", "Pool": "Azure Pipelines" } }, "JavaTestVersion": [ "1.8", "1.11" ], @@ -602,27 +616,28 @@ Given a matrix like below with `JavaTestVersion` marked as a non-sparse paramete } ``` -A matrix with 6 combinations will be generated: A sparse matrix of Agent, AZURE_TEST_HTTP_CLIENTS and ArmTemplateParameters -(3 total combinations) will be multiplied by the two `JavaTestVersion` parameters `1.8` and `1.11`. +A matrix with 6 combinations will be generated: A sparse matrix of `Agent`, `AZURE_TEST_HTTP_CLIENTS` and `ArmTemplateParameters` +(3 total combinations) will be multiplied by the two `JavaTestVersion` parameter values of `1.8` and `1.11`. -NOTE: NonSparseParameters are also applied when generating an imported matrix. +NOTE: `NonSparseParameters` are also applied when generating an imported matrix. -#### Under the hood +## Under the hood -The script generates an N-dimensional matrix with dimensions equal to the parameter array lengths. For example, -the below config would generate a 2x2x1x1x1 matrix (five-dimensional): +The [`Create-JobMatrix.ps1`](https://github.com/Azure/azure-sdk-tools/blob/main/eng/common/scripts/job-matrix/Create-JobMatrix.ps1) script generates an N-dimensional matrix with dimensions equal to the parameter array lengths. +For example, the below config would generate a 2x2x1x1x1 matrix (five-dimensional): ``` json "matrix": { - "framework": [ "net461", "netcoreapp2.1" ], + "framework": [ "net461", "net6.0" ], "additionalTestArguments": [ "", "/p:SuperTest=true" ] - "pool": [ "ubuntu-18.04" ], - "container": [ "ubuntu-18.04" ], + "pool": [ "ubuntu-22.04" ], + "container": [ "ubuntu-22.04" ], "testMode": [ "Record" ] } ``` -The matrix is stored as a one-dimensional array, with a row-major indexing scheme (e.g. `(2, 1, 0, 1, 0)`). +The matrix is stored as a one-dimensional array, with a [row-major](https://en.wikipedia.org/wiki/Row-_and_column-major_order) +indexing scheme (e.g. `(2, 1, 0, 1, 0)`). ## Testing