Set proper exit code on dotnet list package --vulnerable to enable usage in CI pipelines

[rajbos]

Running dotnet list package --vulnerable does not use the exit code to indicate issues. I'd think vulnerable packages with medium / high severity would at least be returned with an exitcode <> 0.

Usage example:

I want to include this command in our CI pipeline to enable a shift left mentality and warn us earlier about possible vulnerabilities. Currently the only option I see is parsing the result for something like has the following vulnerable packages.

Alternatively I can image adding a parameter to return either a number of vulnerable packages (that we can then test to larger then 0) or a json object we can parse the right way to look for the numbers per severity.

I've been searching the source code but could not find where to look for the exit code setup 😁 .

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[thefringeninja]

We're currently using dotnet-retire in our build pipeline. The author is no longer maintaining the list of vulnerable packages and instead suggests we use this (RetireNet/dotnet-retire#75). However, as dotnet list packages --vulnerable returns 0 when it finds vulnerable packages, this doesn't really work in CI without hacks.

@rajbos this feature was added in #13318. It looks like this is just being forwarded to NuGet.

[omajid]

Comparing this with npm, npm audit sets an appropriate exit code if it finds vulnerable packages with a certain class of vulnerabilities: https://docs.npmjs.com/cli/v8/commands/npm-audit#exit-code

[PSanetra]

This was quite unexpected. We will now check the output for has the following vulnerable packages, but fingers crossed that this output does not change in future versions. 🤞

We will now check the output for the absence of has no vulnerable packages. This is more resilient against future changes, but you have to ensure that you check only one project at a time. Otherwise it will not fail if any project has no vulnerable dependencies.

[omeryesil]

This is really disappointing, I can't really recall but there was another dotnet call that really doesn't work well with DevOps scripts. In today's world the automation capability must be a first class citizen.

@PSanetra , about covering multiple projects, we can check has the following vulnerable packages instead of has no vulnerable packages. If there is any vulnerabilities found in a solution we will catch it..

[PSanetra]

@omeryesil The problem with checking for has the following vulnerable packages is that our script may break silently if the phrase changes with a new dotnet version.

[omeryesil]

@PSanetra totally agree with you, but i think it is the only way at the moment unless we use another tool like sonarqube, synk, etc..

[mrjoops]

Hi,

As a workaround, I use these lines to make my (linux-based) CI work:

dotnet list package --vulnerable --include-transitive | tee vulnerable.out
test `grep -cm 1 'has the following vulnerable packages' vulnerable.out` = 0

So yes, it may break with future dotnet versions but since I'm very optimistic, I hope dotnet will implement a correct return code so my added line will never be executed 😉.

[aortiz-msft]

@JonDouglas - This sounds related to the "nuget audit" command that you are thinking of.

[JonDouglas]

Hi all,

I wrote a proposal for this sometime last year I published a couple months ago. If this would fill the gap, please upvote or comment appropriately

NuGet/Home#11549

Thanks!

[ghost]

Thanks for creating this issue! We believe this issue is related to NuGet tooling, which is maintained by the NuGet team. Thus, we closed this one and encourage you to raise this issue in the NuGet repository instead. Don’t forget to check out NuGet’s contributing guide before submitting an issue!

If you believe this issue was closed out of error, please comment to let us know.

Happy Coding!

[erdembayar]

Similar issue opened with NuGet.Client: NuGet/Home#11781

[Evertvdw]

To add to this discussion, I found this issue and adopted @mrjoops answer a bit. The echo parts are specific for Azure pipelines, but it might be helpful to others:

dotnet restore
dotnet list package --vulnerable --include-transitive | tee vulnerable.out
vulnCritical=`grep -o -i 'Critical' vulnerable.out | wc -l`
vulnHigh=`grep -o -i 'High' vulnerable.out | wc -l`
vulnModerate=`grep -o -i 'Moderate' vulnerable.out | wc -l`
vulnLow=`grep -o -i 'Low' vulnerable.out | wc -l`
vulnTotal=`grep -o -i 'Critical\|High\|Moderate\|Low' vulnerable.out | wc -l`
if [[ $vulnTotal != 0 ]]; then
  echo "##vso[task.logissue type=error;]There are vulnarabilities found: Critical($vulnCritical) - High($vulnHigh) - Moderate($vulnModerate) - Low($vulnLow)"
  echo "##vso[task.complete result=Failed;]Text"
fi