Code Coverage doesn't include projects not referenced by Tests Projects

[luiz-soares]

Description

I noticed that projects which are not referenced by Tests projects are not included in the Coverage results, is there are way to get them included? It's important to include them because that does change overall percentage.

I'm creating this since https://stackoverflow.com/questions/42013013/analyze-code-coverage-with-visual-studio-skips-some-projects has not been answered, so hopefully you can help.

I've tried many different configurations in my .runnsettings, including a recursive ModulePaths/IncludeDirectories and specifically including the missing DLL.

I've been reading the docs but didn't find any settings which could make it, did I miss anything?

Thanks!

Steps to reproduce

Solution.sln (C:\source)
ProjectA
ProjectB
TestProject
References ProjectA only

coverage.runsettings:

<RunSettings>
  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
        <Configuration>
          <CodeCoverage>
            <Sources>
	      <Include>
	        <Source>C:\\source\\.*</Source>
	      </Include>
            </Sources>
            <UseVerifiableInstrumentation>False</UseVerifiableInstrumentation>
            <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
            <CollectFromChildProcesses>True</CollectFromChildProcesses>
            <CollectAspDotNet>False</CollectAspDotNet>
          </CodeCoverage>
        </Configuration>
      </DataCollector>
    </DataCollectors>
  </DataCollectionRunSettings>
  <RunConfiguration>
    <MaxCpuCount>0</MaxCpuCount>
    <BatchSize>1000</BatchSize>
    <ResultsDirectory>C:\results</ResultsDirectory>
  </RunConfiguration>
</RunSettings>

Once solution is built, I can see all DLLs and PDBs in their own files, and in TestsProject bin's folder only ProjectA.dll/pdb is there as expected.

Then run vstest

vstest.console.exe "C:\sources\TestProject\bin\release\TestProject.dll" /Settings:"C:\coverage.runsettings" /Logger:"trx" /TestAdapterPath:"c:\Source"

(don't think Logger and TestAdapterPath are important but that is how I've been running)

Expected behavior

.coverage included ProjectB on it, stating 0 covered blocks as that will for sure change the overall percentage

Actual behavior

ProjectB is not included, only ProjectA and TestProjects are present .

Diagnostic logs

I don't have a diag I can share now as I'm building some real projects, if really needed I can create a test solution and do it.

Environment

Microsoft (R) Test Execution Command Line Tool Version 17.0.0

[jakubch1]

@luiz-soares thank you for reporting this. Currently it works in dynamic way. So if dll/exe is not loaded by any test project there will be no coverage for it. Manipulating runsettings will not help. I think your suggestion is valid and we will consider it.

[luiz-soares]

Just found out a workaround, just provide all DLL to vstest even if it contains no tests.

All our DLLs are prefixed so it's easy to find them among all other DLLs, .runsettings then only include the stuff whose source is inside current project source folder, that excludes DLLs still built by us but just referenced by current project.

My pipelines runs on devops and vstest task takes globbing patterns which allow me to easily do that.

So running vstest like this do the work:

vstest.console.exe "C:\sources\TestProject\bin\release\TestProject.dll" "C:\sources\TestProject\bin\release\ProjectA.dll" "C:\sources\TestProject\bin\release\ProjectB.dll" /Settings:"C:\coverage.runsettings" /Logger:"trx" /TestAdapterPath:"c:\Source"