Add guidance for adopting CodeQL

Documentation/CodeQLGuidance.md

@@ -0,0 +1,204 @@
+# CodeQL Guidance
+
+## Background
+
+CodeQL is a code analysis platform owned by Semmle, now a subsidary of GitHub. It provides value by using extractors to construct a database representing the codebase, then providing a query language to perform sematic analysis. CodeQL instruments the build of compiled languages, and directly analyzes source code for interpreted languages. 
+
+CodeQL is required as part of Microsoft's Security Developent Lifecycle (SDL) requirements. .NET Engineering Services supports CodeQL via the Guardian toolset with scan results published to Trust Services Automation (TSA). 
+
+CodeQL adds a significant time to builds. We therefore recommend creating a new, seperate pipeline instead of incorporating CodeQL scans into existing PR or testing pipelines. 
+
+## TL;DR: Quickstart
+
+If using Arcade, 
+
+1. Ensure your repository has the latest Arcade version
+2. Copy Arcade's CodeQL pipeline definition file [`azure-pipelines-codeql.yml`](https://github.dev/dotnet/arcade/blob/ecf92bbdc4354adc1960fa53808e87d91624f9b5/azure-pipelines-codeql.yml) to your repository
+3. Modify the pipeline definition to work for your repository's needs
+   - For projects using compiled languages, like C#, update the `buildCommand`.
+   - Update the `language` parameter to match your project's language 
+   - For projects with multiple languages, duplicate the Job for each. 
+4. Create a new Pipeline executing this newly-created definition. 
+
+If using the SDL Azure DevOps Extension,
+
+1. Copy the example pipeline definition from [Use with SDL Azure DevOps Extension](#Use-with-SDL-Azure-DevOps-Extension) section to your repository.
+2. Modify the example to work for your repository's needs
+   - For projects using compiled languages, like C#, update the `buildCommand`.
+   - Update the `language` parameter to match your project's language 
+   - For projects with multiple languages, duplicate the Job for each. 
+3. Create a new Pipeline executing this newly-created definition.
+
+## Use with Arcade
+
+Arcade version (arcade version) provides a new Job template "[codeql-build.yml](https://github.com/dotnet/arcade/blob/main/eng/common/templates/jobs/codeql-build.yml)" and step template "[execute-codeql.yml](https://github.dev/dotnet/arcade/blob/main/eng/common/templates/steps/execute-codeql.yml)". It extends Arcade's existing use of Guardian to provide SDL tooling and workflow.
+
+A working example pipeline defintion follows.
+
+```yaml
+#Include variables for SDL/TSA publishing
+variables:
+  - name: _TeamName
+    value: DotNetCore
+  - group: SDL_Settings
+
+# Execute only on a schedule, once each week
+trigger: none
+
+schedules:
+  - cron: 0 12 * * 1
+    displayName: Weekly Monday CodeQL/Semmle run
+    branches:
+      include:
+      - main
+    always: true
+
+# One stage, one job: just CodeQL
+stages:
+- stage: build
+  displayName: Build
+  jobs:
+  - template: /eng/common/templates/jobs/codeql-build.yml
+    parameters:
+      jobs:
+      - job: Windows_NT_CSharp
+        timeoutInMinutes: 90
+        pool:
+          name: NetCore1ESPool-Internal
+          demands: ImageOverride -equals Build.Server.Amd64.VS2019
+
+        steps:
+        - checkout: self
+          clean: true
+
+        - template: /eng/common/templates/steps/execute-codeql.yml
+          parameters:
+            executeAllSdlToolsScript: 'eng/common/sdl/execute-all-sdl-tools.ps1'
+            buildCommands: 'build.cmd -configuration Release -ci -prepareMachine'
+            language: csharp
+            additionalParameters: '-SourceToolsList @("semmle")
+            -TsaInstanceURL $(_TsaInstanceURL)
+            -TsaProjectName $(_TsaProjectName)
+            -TsaNotificationEmail $(_TsaNotificationEmail)
+            -TsaCodebaseAdmin $(_TsaCodebaseAdmin)
+            -TsaBugAreaPath $(_TsaBugAreaPath)
+            -TsaIterationPath $(_TsaIterationPath)
+            -TsaRepositoryName "dotnet-release"
+            -TsaCodebaseName "dotnet-release"
+            -TsaPublish $True'
+```
+
+Much of this template can be used as-is for most repositories. The important elements for customizatino are the input parameters to the `execute-codeql` template. 
+
+- `buildCommands`: For compiled languages, the command to build the project. For interpreted languages, this may be excluded. 
+- `language`: The language target for analysis (as known by CodeQL)
+- `additionalParameters`: These are the typical TSA configuration options usually already populated for .NET projects. Ensure that `TsaRepositoryName` and `TsaCodebaseName` are correct for your repository.
+
+For more information on SDL/Guardian in Arcade, see [How To Add SDL Run To Pipeline](https://github.dev/dotnet/arcade/blob/main/Documentation/HowToAddSDLRunToPipeline.md).
+
+## Use with SDL Azure DevOps Extension
+
+The [SDL Azure DevOps Extension](https://aka.ms/sdtvstsext) provides access to Guardian tools through custom pipeline tasks.
+
+```yaml
+variables:
+  LGTM.UploadSnapshot: false  # Do not use the LGTM service
+  Semmle.SkipAnalysis: false  # Do use the Guardian-included alerts
+
+# Execute only on a schedule, once each week
+trigger: none
+
+schedules:
+  - cron: 0 12 * * 1
+    displayName: Weekly Monday CodeQL/Semmle run
+    branches:
+      include:
+      - main
+    always: true
+
+# One stage, one job: just CodeQL
+stages:
+- stage: CodeQL
+  displayName: CodeQL
+
+  jobs:
+  - job: CSharp
+    timeoutInMinutes: 90
+    pool: 
+      name: NetCore1ESPool-Internal
+      demands: ImageOverride -equals Build.Server.Amd64.VS2019
+      displayName: "CodeQL: C#"
+
+    steps:      
+    # Run the CodeQL (aka Semmle) scan
+    - task: Semmle@1
+      displayName: 'Analyze csharp projects'
+      inputs:
+        toolVersion: 'Latest'
+        sourceCodeDirectory: '$(Build.SourcesDirectory)'
+        language: 'csharp'
+        buildCommandsString: 'build.cmd -configuration Release -prepareMachine /p:Test=false /P:Sign=false'
+        querySuite: 'Required'
+        timeout: '1800'
+        ram: '16384'
+        addProjectDirToScanningExclusionList: true
+
+    # Generate SARIF report
+    - task: SdtReport@2 
+      displayName: Generate Reports
+      inputs:
+        GdnExportAllTools: true
+
+    # Save the report for discovery in the Azure DevOps UI
+    - task: PublishSecurityAnalysisLogs@3
+      displayName: Publish Logs
+      inputs:
+        ArtifactName: CodeAnalysisLogs
+        ArtifactType: Container
+
+    # If alerts are found, fail the build
+    - task: PostAnalysis@2
+      displayName: Post Analysis (Build Break)
+      inputs:
+        GdnBreakAllTools: true
+```
+
+Most of this example may be used as-is by most repositories. The important elements to examine are:
+
+- `language`: The language mode to operate in. 
+- `buildCommandsString`: For compile languages, the command to execute to build the project. For interpreted languages this may be excluded. 
+
+If scanning more than one language, duplicate the Job for each.
+
+More information about these tasks, including a reference for YAML elements, is available from the individual task sites.
+
+- [CodeQL Build Task](https://www.1eswiki.com/wiki/CodeQL_Build_Task) aka Semmle
+- [Security Analysis Report Build Task](https://www.1eswiki.com/wiki/Security_Analysis_Report_Build_Task) aka SdtReport
+- [Publish Security Analysis Logs Build Task](https://www.1eswiki.com/wiki/Publish_Security_Analysis_Logs_Build_Task) aka PublishSecurityAnalysisLogs
+- [Post Analysis Build Task](https://www.1eswiki.com/wiki/Post_Analysis_Build_Task) aka PostAnalysis
+
+For more documentation on the SDL Extension, see [Secure Development Tools Extension For Azure DevOps](https://aka.ms/sdt-AzDevOps)
+
+## Alert suppression
+
+Supression may be done using inline comments (in whatever comment form is appropriate for the language). The comment must appear on the same line as the alert, or the first line if the alert spans multiple lines. 
+
+A suppression comment is made of:
+
+1. The string "lgtm[^1]" (case insensitive)
+2. A query ID surrounded by square brackets
+3. A justification string of at least 25 characters
+
+For example, in C#,
+
+```cs
+// lgtm [cs/weak-crypto] Algorithm needed per standard and contained safely here
+```
+
+Language-specific examples and some variations may be found in LGTM's [Alert Suppression](https://lgtm.com/help/lgtm/alert-suppression) document.
+
+[^1]: At the time of this writing, tools expect `lgtm` instead of `codeql`. This may change as versions evolve and the new name propogates.
+
+## Further Reading
+
+GitHub's official CodeQL documentation: [CodeQL documentation](https://codeql.github.com/docs)