Moq.dll shows up as uncovered code in Visual Studio as of v4.16

[jenkinsonc]

I previously was using v4.14.7 and recently upgraded to v4.16. When I run my unit tests as "Analyze Code Coverage", moq.dll shows up as one of the assemblies, with 1220/6719 blocks covered. While I can create a CodeCoverage.runsettings file to specifically exclude it and change our pipeline to exclude it as well, the desired behavior is that of the prior version of Moq (and other libraries I include from Nuget).

[stakx]

Well, I don't know why code analysis would analyze a third-party assembly in the first place, instead of just your source code. I dare say this is mostly an issue with your project's configuration, and not with Moq.

What would one have to do to reproduce this?

[jenkinsonc]

It's very possible is project configuration, but like I said it was ignoring moq in version 4.14.7 and now includes it, and doesn't include other third party assemblies even after updating them. I'm in Visual Studio 16.8.1, my unit test project is a .NET Core 3.1 project using the Microsoft.VisualStudio.TestTools.UnitTesting library for testing. I've got Moq added only to my unit test project.

If I use the Test Explorer to Analyze Code Coverage with Moq 4.14.7, I only see my assemblies tested. I am using the sample CodeCoverage.runsettings file found here with the provided ModulePaths, Functions, and PublicKey sections removed, and my unit test ModulePath added. I can paste the contents if you want. It has no explicit Includes, only Excludes.

Then I upgrade Moq to 4.16, Analyze Code Coverage again, Moq appears in the list. If I set Visual Studio to not use the runsettings file, Moq is not included in the output set on either version. That led me to believe it's one of these settings:

`

        <UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>

        <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>

        <CollectFromChildProcesses>True</CollectFromChildProcesses>

`
However, setting them all to False has no change on the output; moq.dll still appears. Removing the CodeCoverage.runsettings file does hide moq.dll from the results, but of course adds the unit test assembly I'm trying to hide. My workaround so far has been to add Moq to the ModulePath exclusion list:

<ModulePaths> <Exclude> <ModulePath>.*\moq.dll$</ModulePath> </Exclude> </ModulePaths>

[stakx]

I'm pretty sure this change stems from the change made in #1098 with regard to how we distribute Moq's PDB file; see #1098 (comment).

I think it is reasonable to have to configure code analysis for a specific project to a certain extent. That being said, I find this behaviour of VS code analysis somewhat unfortunate, I would have expected there to be some kind of "Just My Code" switch just like there is one for debugging.

@kzu if this is indeed caused by our including the PDB in the main NuGet, what do you make of this? Were you previously aware of that side effect?

[siprbaum]

Could it be that "Just my Code" is not working at and therefore also the coverage is wrong?
At least there is some issue in nunit/nunit#3745 which tells that debuging a unit test with "Just my Code" enabled still steps into the nunit framework code. The issue seems to be related to embedding the symbols via

    <DebugType>embedded</DebugType>

[stakx]

@siprbaum If I understood the OP correctly, this is about code analysis, not about debugging. I only mentioned the "Just My Code" setting (which AFAICT targets debugging exclusively) because its analog for code analysis would appear to be really handy in the OP's situation.

[siprbaum]

My point here was that Visual Studio has - according to the issue I referenced - a problem when debugging that it considers assemblies with embedded symbols as your own code. If the same logic to detect if "is this your code" is used when gathering the code coverage, it would explain why it gets confused and no longer filters out the Moq assembly from the coverage results.

But I agree that this is a shot in the dark.

There is a way to disable the coverage for certain classes, but I have no idea if this is even practical at all.

[stakx]

My apologies @siprbaum for misunderstanding your post. That's a reasonable guess.

[kzu]

hm... I'm not aware of symbols embedding being a cause for a difference in code coverage analysis 🤔

[stakx]

@kzu I suppose you don't want to see us going back to .snupkg? 😃

[kzu]

Oh, no plz! FWIW, excluding an assembly from codeanalysis results is quite trivial. I don't know why anyone consuming Moq would want to see code coverage for it alongside their own assemblies.

So at most I'd just document the known issue and perhaps point to the docs on how to exclude assemblies from results?

[stakx]

I'm going to close this without taking any further action. @jenkinsonc if you do configure code analysis to exclude Moq, and you feel the procedure might be helpful to others, feel free to document it in the wiki (e.g. here, for the time being, until we have a more official place for documentation).