Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing coverage in 1.7.0 #852

Closed
JustinBerger opened this issue May 20, 2020 · 9 comments
Closed

Missing coverage in 1.7.0 #852

JustinBerger opened this issue May 20, 2020 · 9 comments
Labels
duplicate This issue or pull request already exists waiting for customer Waiting for customer action

Comments

@JustinBerger
Copy link

JustinBerger commented May 20, 2020

In a .net core project running using coverlet.console 1.6.0 we get these results.

+--------------------------------------+--------+--------+--------+
| Module                               | Line   | Branch | Method |
+--------------------------------------+--------+--------+--------+
| namespace            | 97.53% | 99.34% | 100%   |
+--------------------------------------+--------+--------+--------+
| namespace.ViewModels | 100%   | 100%   | 100%   |
+--------------------------------------+--------+--------+--------+

But when running in 1.7.0 (and 1.7.1) we get

+--------------------------------------+--------+--------+--------+
| Module                               | Line   | Branch | Method |
+--------------------------------------+--------+--------+--------+
| namespace            | 99.11% | 75.75% | 95.53% |
+--------------------------------------+--------+--------+--------+
| namespace.ViewModels | 87.23% | 100%   | 87.23% |
+--------------------------------------+--------+--------+--------+

Running on windows with this command
coverlet .\tests\UnitTests\bin\Debug\netcoreapp3.1\namespace.UnitTests.dll --target "dotnet" --targetargs "test .\tests\UnitTests --no-build" --format opencover --output "c:\dev\coverage\namespace\" --exclude '[coverlet.*]*' --exclude '[NUnit3.*]*' --exclude '[Refit]*' --exclude '[StackExchange.*]*' --exclude '[Refit.*]*' --exclude '[System.*]*' --exclude '[Pipelines.*]*' --include '[*]*'

When we run Report generator against the opencover report, it shows whole methods as if they were set to ignore coverage, rather than being uncovered.
The tests cover those methods (though not completely).

I don't yet have a sample app I can reproduce the results with.

@MarcoRossignoli
Copy link
Collaborator

Do you have [ExcludeFromCoverage] somewhere in your code, we fixed a bug on that, there was some place where it wasn't honored so with new version coverage could drop.
It's hard without a repro o a piece of code to compare, usually could be an issue if methods is async state machine(async/await), has got strange lambda expression(LINQ) or as I've said skip some attribute decorated methods.

@MarcoRossignoli MarcoRossignoli added the needs more info More details are needed label May 21, 2020
@JustinBerger
Copy link
Author

We do have exclude tags elsewhere in the project. Those seem to have been handled correctly. It just completely lost whole controller methods. I'll try to work up an example when I have a moment.

@MarcoRossignoli
Copy link
Collaborator

Also can you try to use collectors integration?Msbuild and .net tool at the moment suffer of some issue related to vstest behaviour https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/VSTestIntegration.md

@MarcoRossignoli MarcoRossignoli added waiting for customer Waiting for customer action and removed needs more info More details are needed labels May 23, 2020
@askpt
Copy link

askpt commented Jun 9, 2020

I updated the coverlet.collectors to the latest version and I'm hitting some issues.
My project is dotnet core latest version and I have some [ExcludeFromCodeCoverage].

The report in the VSTS reports that most of the async methods are Not Coverable while the other ones are. Using Rider I'm able to get coverage of the entire solution and green lines on async methods.

@MarcoRossignoli
Copy link
Collaborator

@askpt can you attach your command line?Could be an issue with some attribute filters maybe.

@askpt
Copy link

askpt commented Jun 9, 2020

@MarcoRossignoli sure! Here is my runsettings:

<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
    <DataCollectionRunSettings>
        <DataCollectors>
            <DataCollector friendlyName="XPlat code coverage">
                <Configuration>
                    <Format>opencover,cobertura</Format>
                    <ExcludeByAttribute>Obsolete,GeneratedCodeAttribute,CompilerGeneratedAttribute</ExcludeByAttribute>
                    <SingleHit>false</SingleHit>
                    <UseSourceLink>true</UseSourceLink>
                    <IncludeTestAssembly>false</IncludeTestAssembly>
                </Configuration>
            </DataCollector>
        </DataCollectors>
    </DataCollectionRunSettings>
</RunSettings>

VSTS Task:

- task: DotNetCoreCLI@2
      displayName: Test Api
      inputs:
        command: 'test'
        projects: '**/*.Tests.csproj'
        arguments: '--configuration $(buildConfiguration) --no-restore --collect:"XPlat Code Coverage" --settings coverlet.runsettings'
        verbosityRestore: 'Normal'
        publishTestResults: true

@MarcoRossignoli
Copy link
Collaborator

MarcoRossignoli commented Jun 9, 2020

Obsolete,GeneratedCodeAttribute,CompilerGeneratedAttribute

You cannot use GeneratedCodeAttribute because for every async method roslyn compile a state maching with attribute [GeneratedCodeAttribute] there was a bug in filtering in past and this wasn't honored, now we fixed that.

dup of #794 (comment)

@MarcoRossignoli MarcoRossignoli added the duplicate This issue or pull request already exists label Jun 9, 2020
@MarcoRossignoli
Copy link
Collaborator

@JustinBerger any news?

@MarcoRossignoli
Copy link
Collaborator

Closed for stale conversation, feel free to re-open if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists waiting for customer Waiting for customer action
Projects
None yet
Development

No branches or pull requests

3 participants