Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Global coverage threshold should not exclude glob based thresholds #6165

Closed
5 of 6 tasks
thor-juhasz opened this issue Jul 18, 2024 · 1 comment · Fixed by #6172
Closed
5 of 6 tasks

Global coverage threshold should not exclude glob based thresholds #6165

thor-juhasz opened this issue Jul 18, 2024 · 1 comment · Fixed by #6172
Labels
feat: coverage Issues and PRs related to the coverage feature p3-minor-bug An edge case that only affects very specific usage (priority)

Comments

@thor-juhasz
Copy link
Contributor

thor-juhasz commented Jul 18, 2024

Describe the bug

When using both global coverage thresholds and pattern based coverage thresholds, the "All files" tally reports correct numbers.

Setting the global thresholds based on this for the next run will fail the test run since the files matched by patterns are deducted from the global threshold numbers before checking if they pass.

Reproduction

Use this in test config:

coverage: {
    reporter: ['text', 'html'],
    thresholds: {
        lines: 50,

        'src/b.ts': {
            lines: 100,
        }
    },
},

src/a.ts:

export function add(a: number, b: number): number {
    return a + b;
}

src/b.ts:

export function subtract(a: number, b: number): number {
    return a - b;
}

Cover src/a.ts with a test and then run the tests:

test('adds numbers', () => {
    expect(add(1, 2)).toBe(3);
});

You will see that it fails on global threshold, even though we have covered 100% of a.ts (and that passes), and 0% of b.ts, which results in a total of 50% coverage. And that is what the text reporter tells us.

However, the test run is failed as it only sees b.ts for the global thresholds, confusing users, implying the coverage numbers are somehow off.

System Info

System:
    OS: Linux 6.5 Ubuntu 22.04.4 LTS 22.04.4 LTS (Jammy Jellyfish)
    CPU: (20) x64 12th Gen Intel(R) Core(TM) i7-12700K
    Memory: 29.25 GB / 62.54 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 20.15.0 - ~/.nvm/versions/node/v20.15.0/bin/node
    Yarn: 4.3.1 - ~/.nvm/versions/node/v20.15.0/bin/yarn
    npm: 10.7.0 - ~/.nvm/versions/node/v20.15.0/bin/npm
  Browsers:
    Chrome: 126.0.6478.126
  npmPackages:
    @vitejs/plugin-basic-ssl: ^1.1.0 => 1.1.0 
    @vitejs/plugin-vue: ^5.0.5 => 5.0.5 
    @vitest/coverage-istanbul: ^2.0.3 => 2.0.3 
    @vitest/ui: 2.0.3 => 2.0.3 
    vite: ^5.0.0 => 5.3.3 
    vitest: ^2.0.3 => 2.0.3

Used Package Manager

yarn

Validations

@AriPerkkio
Copy link
Member

During the implementation of glob based thresholds I ran into this as well: #4442 (comment). I didn't know whether this is a bug or not, so I followed Jest's way and implented the same logic. But now I think this is bug indeed.

The related code is here and there's also comment describing the current behavior:

// Global threshold is for all files that were not included by glob patterns
for (const file of files.filter(
file => !filesMatchedByGlobs.includes(file),
)) {
const fileCoverage = coverageMap.fileCoverageFor(file)
globalCoverageMap.addFileCoverage(fileCoverage)
}

@AriPerkkio AriPerkkio added feat: coverage Issues and PRs related to the coverage feature p3-minor-bug An edge case that only affects very specific usage (priority) and removed pending triage labels Jul 19, 2024
thor-juhasz added a commit to thor-juhasz/vitest that referenced this issue Jul 19, 2024
thor-juhasz added a commit to thor-juhasz/vitest that referenced this issue Jul 19, 2024
thor-juhasz added a commit to thor-juhasz/vitest that referenced this issue Jul 19, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Aug 5, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feat: coverage Issues and PRs related to the coverage feature p3-minor-bug An edge case that only affects very specific usage (priority)
Projects
None yet
2 participants