Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding command policy to multitool #2118

Merged
merged 4 commits into from
Oct 21, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/ReleaseHistory.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
* FEATURE: The Multitool `query` command can now evaluate properties in the result and rule property bags, for example `sarif query "properties.confidence:f > 0.95 AND rule.properties.category == 'security'"`
* FEATURE: The validation rule `SARIF1004.ExpressUriBaseIdsCorrectly` now verifies that if an `artifactLocation.uri` is a relative reference, it does not begin with a slash. [#2090](https://github.com/microsoft/sarif-sdk/issues/2090)
* BUGFIX: GitHub policy should not turn off any note level rules. [#2089](https://github.com/microsoft/sarif-sdk/issues/2089)
* FEATURE: Add `apply-policy` command to Multitool. [#2118](https://github.com/microsoft/sarif-sdk/pull/2118)

## **v2.3.6** [Sdk](https://www.nuget.org/packages/Sarif.Sdk/2.3.6) | [Driver](https://www.nuget.org/packages/Sarif.Driver/2.3.6) | [Converters](https://www.nuget.org/packages/Sarif.Converters/2.3.6) | [Multitool](https://www.nuget.org/packages/Sarif.Multitool/2.3.6) | [Multitool Library](https://www.nuget.org/packages/Sarif.Multitool.Library/2.3.6)
* BUGFIX: Restore multitool client app package build.
Expand Down
65 changes: 65 additions & 0 deletions src/Sarif.Multitool.Library/ApplyPolicyCommand.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System;
using System.Diagnostics;
using Microsoft.CodeAnalysis.Sarif.Driver;
using Newtonsoft.Json;

namespace Microsoft.CodeAnalysis.Sarif.Multitool
{
public class ApplyPolicyCommand : CommandBase
{
private readonly IFileSystem _fileSystem;

public ApplyPolicyCommand(IFileSystem fileSystem = null)
{
_fileSystem = fileSystem ?? FileSystem.Instance;
}

public int Run(ApplyPolicyOptions applyPolicyOptions)
{
try
{
Console.WriteLine($"Applying policy '{applyPolicyOptions.InputFilePath}' => '{applyPolicyOptions.OutputFilePath}'...");
Stopwatch w = Stopwatch.StartNew();

bool valid = ValidateOptions(applyPolicyOptions);
if (!valid) { return FAILURE; }

SarifLog actualLog = ReadSarifFile<SarifLog>(_fileSystem, applyPolicyOptions.InputFilePath);

actualLog.ApplyPolicies();

string fileName = CommandUtilities.GetTransformedOutputFileName(applyPolicyOptions);

Formatting formatting = applyPolicyOptions.PrettyPrint
? Formatting.Indented
: Formatting.None;
Copy link

@ghost ghost Oct 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You know, we have this transformation from "common options" to Newtonsoft Formatting in many places. This could be an extension method on CommonOptions (it would be called like this: applyPolicyOptions.GetFormatting(). Ask @michaelcfanning if this is a good time to do that. #Resolved

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can do a separate pr for that. So i can tackle all places, what do you think?


In reply to: 506574244 [](ancestors = 506574244)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, that's fine.


In reply to: 506659476 [](ancestors = 506659476,506574244)

Copy link
Collaborator Author

@eddynaka eddynaka Oct 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new pr with this: #2121 #Resolved


WriteSarifFile(_fileSystem, actualLog, fileName, formatting);

w.Stop();
Console.WriteLine($"Rewrite completed in {w.Elapsed}.");
}
catch (Exception ex)
{
Console.WriteLine(ex);
return FAILURE;
}

return SUCCESS;
}

private bool ValidateOptions(ApplyPolicyOptions applyPolicyOptions)
{
bool valid = true;

valid &= applyPolicyOptions.Validate();

valid &= DriverUtilities.ReportWhetherOutputFileCanBeCreated(applyPolicyOptions.OutputFilePath, applyPolicyOptions.Force, _fileSystem);

return valid;
}
}
}
13 changes: 13 additions & 0 deletions src/Sarif.Multitool.Library/ApplyPolicyOptions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using CommandLine;
using Microsoft.CodeAnalysis.Sarif.Driver;

namespace Microsoft.CodeAnalysis.Sarif.Multitool
{
[Verb("apply-policy", HelpText = "Apply policies from SARIF log.")]
public class ApplyPolicyOptions : SingleFileOptionsBase
{
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

} [](start = 4, length = 1)

Will there in future be an option to specify one out of multiple policies present in the file?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will apply all policies. Should we rename it?


In reply to: 506563829 [](ancestors = 506563829)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to rename. But... ok, this is a "requirements" question that you should discuss with @michaelcfanning. As the code stands, you apply policies in order, and if two policies refer to the same rule, the last one wins (as your test verifies). Is Michael sure that's what we want?


In reply to: 506565036 [](ancestors = 506565036,506563829)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely what we specified for now.


In reply to: 506571247 [](ancestors = 506571247,506565036,506563829)

}
2 changes: 2 additions & 0 deletions src/Sarif.Multitool/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ internal static class Program
public static int Main(string[] args)
{
return Parser.Default.ParseArguments<
ApplyPolicyOptions,
ExportValidationConfigurationOptions,
ExportValidationDocumentationOptions,
ExportValidationRulesMetadataOptions,
Expand All @@ -29,6 +30,7 @@ public static int Main(string[] args)
ResultMatchSetOptions,
FileWorkItemsOptions>(args)
.MapResult(
(ApplyPolicyOptions options) => new ApplyPolicyCommand().Run(options),
(ExportValidationConfigurationOptions options) => new ExportValidationConfigurationCommand().Run(options),
(ExportValidationDocumentationOptions options) => new ExportValidationDocumentationCommand().Run(options),
(ExportValidationRulesMetadataOptions options) => new ExportValidationRulesMetadataCommand().Run(options),
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
// Copyright (c) Microsoft. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System.IO;
eddynaka marked this conversation as resolved.
Show resolved Hide resolved
using FluentAssertions;
using Xunit;

namespace Microsoft.CodeAnalysis.Sarif.Multitool
{
public class ApplyPolicyCommandTests
{
private static readonly ResourceExtractor Extractor = new ResourceExtractor(typeof(ApplyPolicyCommandTests));

[Fact]
public void WhenInputContainsOnePolicy_ShouldSucceed()
{
string path = "WithPolicy.sarif";
File.WriteAllText(path, Extractor.GetResourceText($"ApplyPolicyCommand.{path}"));

// Verify log loads, has correct Result count, and spot check a Result
SarifLog log = ExecuteTest(path);
log.Runs[0].Results.Count.Should().Be(1);
log.Runs[0].Results[0].Level.Should().Be(FailureLevel.Error);
}
Copy link

@ghost ghost Oct 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to delete the files. They are written to the test binaries directory and overwritten on each run (and deleted on a clean build). #Closed


[Fact]
public void WhenInputContainsMultiplePolicies_ShouldApplyPoliciesInOrder()
{
string path = "WithPolicy2.sarif";
File.WriteAllText(path, Extractor.GetResourceText($"ApplyPolicyCommand.{path}"));

// Verify log loads, has correct Result count, and spot check a Result
SarifLog log = ExecuteTest(path);
log.Runs[0].Results.Count.Should().Be(1);
log.Runs[0].Results[0].Level.Should().Be(FailureLevel.Note);
}

private SarifLog ExecuteTest(string path)
{
var options = new ApplyPolicyOptions
{
InputFilePath = path,
OutputFilePath = path,
Force = true
};

// Verify command returned success
int returnCode = new ApplyPolicyCommand().Run(options);
returnCode.Should().Be(0);

// Verify SARIF output log exists
File.Exists(path).Should().BeTrue();

return SarifLog.Load(path);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@
</ItemGroup>

<ItemGroup>
<EmbeddedResource Include="TestData\ApplyPolicyCommand\WithPolicy2.sarif" />
<EmbeddedResource Include="TestData\ApplyPolicyCommand\WithPolicy.sarif" />
<EmbeddedResource Include="TestData\ConvertCommand\SemmleQlSample.csv" />
<EmbeddedResource Include="TestData\ExportRuleDocumentationCommand\ExpectedOutputs\NonStandardMessageStringKey.md" />
<EmbeddedResource Include="TestData\ExportRuleDocumentationCommand\ExpectedOutputs\StandardMessageStringKey.md" />
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
{
"$schema": "http://json.schemastore.org/sarif-2.1.0-rtm.1",
"version": "2.1.0",
"runs": [
{
"tool": {
"driver": {
"name": "Test",
"version": "1.0.0",
"rules": [
{
"id": "TEST0001",
"name": "Test",
"shortDescription": {
"text": "Test description."
},
"messageStrings": {
"default": {
"text": "Test description."
}
}
}
]
}
},
"results": [
{
"ruleId": "TEST0001",
"ruleIndex": 0,
"message": {
"text": "Test text."
}
}
],
"columnKind": "utf16CodeUnits",
"policies": [
{
"name": "TEST Policy",
"version": "1.0.0",
"rules": [
{
"id": "TEST0001",
"defaultConfiguration": {
"level": "error"
}
}
]
}
]
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
{
"$schema": "http://json.schemastore.org/sarif-2.1.0-rtm.1",
"version": "2.1.0",
"runs": [
{
"tool": {
"driver": {
"name": "Test",
"version": "1.0.0",
"rules": [
{
"id": "TEST0001",
"name": "Test",
"shortDescription": {
"text": "Test description."
},
"messageStrings": {
"default": {
"text": "Test description."
}
}
}
]
}
},
"results": [
{
"ruleId": "TEST0001",
"ruleIndex": 0,
"message": {
"text": "Test text."
}
}
],
"columnKind": "utf16CodeUnits",
"policies": [
{
"name": "TEST Policy",
"version": "1.0.0",
"rules": [
{
"id": "TEST0001",
"defaultConfiguration": {
"level": "error"
}
}
]
},
{
"name": "TEST Policy 2",
"version": "1.0.0",
"rules": [
{
"id": "TEST0001",
"defaultConfiguration": {
"level": "note"
}
}
]
}
]
}
]
}