Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!(coverage): glob based coverage thresholds #4442

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 70 additions & 31 deletions docs/config/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -1181,70 +1181,109 @@ Collect coverage of files outside the [project `root`](https://vitest.dev/config

Do not show files with 100% statement, branch, and function coverage.

#### coverage.perFile
#### coverage.thresholds

- **Type:** `boolean`
- **Default:** `false`
- **Available for providers:** `'v8' | 'istanbul'`
- **CLI:** `--coverage.perFile`, `--coverage.perFile=false`

Check thresholds per file.
See `lines`, `functions`, `branches` and `statements` for the actual thresholds.
Options for coverage thresholds

#### coverage.thresholdAutoUpdate
##### coverage.thresholds.lines

- **Type:** `boolean`
- **Default:** `false`
- **Type:** `number`
- **Available for providers:** `'v8' | 'istanbul'`
- **CLI:** `--coverage.thresholdAutoUpdate=<boolean>`
- **CLI:** `--coverage.thresholds.lines=<number>`

Update threshold values `lines`, `functions`, `branches` and `statements` to configuration file when current coverage is above the configured thresholds.
This option helps to maintain thresholds when coverage is improved.
Global threshold for lines.
See [istanbul documentation](https://github.com/istanbuljs/nyc#coverage-thresholds) for more information.

#### coverage.lines
##### coverage.thresholds.functions

- **Type:** `number`
- **Available for providers:** `'v8' | 'istanbul'`
- **CLI:** `--coverage.lines=<number>`
- **CLI:** `--coverage.thresholds.functions=<number>`

Threshold for lines.
Global threshold for functions.
See [istanbul documentation](https://github.com/istanbuljs/nyc#coverage-thresholds) for more information.

#### coverage.functions
##### coverage.thresholds.branches

- **Type:** `number`
- **Available for providers:** `'v8' | 'istanbul'`
- **CLI:** `--coverage.functions=<number>`
- **CLI:** `--coverage.thresholds.branches=<number>`

Threshold for functions.
Global threshold for branches.
See [istanbul documentation](https://github.com/istanbuljs/nyc#coverage-thresholds) for more information.

#### coverage.branches
##### coverage.thresholds.statements

- **Type:** `number`
- **Available for providers:** `'v8' | 'istanbul'`
- **CLI:** `--coverage.branches=<number>`
- **CLI:** `--coverage.thresholds.statements=<number>`

Threshold for branches.
Global threshold for statements.
See [istanbul documentation](https://github.com/istanbuljs/nyc#coverage-thresholds) for more information.

#### coverage.statements
##### coverage.thresholds.perFile

- **Type:** `number`
- **Type:** `boolean`
- **Default:** `false`
- **Available for providers:** `'v8' | 'istanbul'`
- **CLI:** `--coverage.statements=<number>`
- **CLI:** `--coverage.thresholds.perFile`, `--coverage.thresholds.perFile=false`

Threshold for statements.
See [istanbul documentation](https://github.com/istanbuljs/nyc#coverage-thresholds) for more information.
Check thresholds per file.

#### coverage.100
##### coverage.thresholds.autoUpdate

- **Type:** `boolean`
- **Default:** `false`
- **Available for providers:** `'v8' | 'istanbul'`
- **CLI:** `--coverage.100`, `--coverage.100=false`
- **CLI:** `--coverage.thresholds.autoUpdate=<boolean>`

Update all threshold values `lines`, `functions`, `branches` and `statements` to configuration file when current coverage is above the configured thresholds.
This option helps to maintain thresholds when coverage is improved.

##### coverage.thresholds.100

Shortcut for `--coverage.lines 100 --coverage.functions 100 --coverage.branches 100 --coverage.statements 100`.
- **Type:** `boolean`
- **Default:** `false`
- **Available for providers:** `'v8' | 'istanbul'`
- **CLI:** `--coverage.thresholds.100`, `--coverage.thresholds.100=false`

Sets global thresholds to 100.
Shortcut for `--coverage.thresholds.lines 100 --coverage.thresholds.functions 100 --coverage.thresholds.branches 100 --coverage.thresholds.statements 100`.

##### coverage.thresholds[glob-pattern]

- **Type:** `{ statements?: number functions?: number branches?: number lines?: number }`
- **Default:** `undefined`
- **Available for providers:** `'v8' | 'istanbul'`

Sets thresholds for files matching the glob pattern.

<!-- eslint-skip -->
```ts
{
coverage: {
thresholds: {
// Thresholds for all files
functions: 95,
branches: 70,
// Thresholds for matching glob pattern
'src/utils/**.ts': {
statements: 95,
functions: 90,
branches: 85,
lines: 80,
},
// Files matching this pattern will only have lines thresholds set.
// Global thresholds are not inherited.
'**/math.ts': {
lines: 100,
}
}
}
}
```

#### coverage.ignoreClassMethods

Expand Down
1 change: 1 addition & 0 deletions packages/coverage-istanbul/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
"istanbul-lib-report": "^3.0.1",
"istanbul-lib-source-maps": "^4.0.1",
"istanbul-reports": "^3.1.6",
"magicast": "^0.3.2",
"picocolors": "^1.0.0",
"test-exclude": "^6.0.0"
},
Expand Down
65 changes: 36 additions & 29 deletions packages/coverage-istanbul/src/provider.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import { existsSync, promises as fs } from 'node:fs'
import { existsSync, promises as fs, writeFileSync } from 'node:fs'
import { resolve } from 'pathe'
import type { AfterSuiteRunMeta, CoverageIstanbulOptions, CoverageProvider, ReportContext, ResolvedCoverageOptions, Vitest } from 'vitest'
import { coverageConfigDefaults, defaultExclude, defaultInclude } from 'vitest/config'
import { BaseCoverageProvider } from 'vitest/coverage'
import c from 'picocolors'
import { parseModule } from 'magicast'
import libReport from 'istanbul-lib-report'
import reports from 'istanbul-reports'
import type { CoverageMap, CoverageMapData } from 'istanbul-lib-coverage'
Expand Down Expand Up @@ -65,10 +66,14 @@ export class IstanbulCoverageProvider extends BaseCoverageProvider implements Co
provider: 'istanbul',
reportsDirectory: resolve(ctx.config.root, config.reportsDirectory || coverageConfigDefaults.reportsDirectory),
reporter: this.resolveReporters(config.reporter || coverageConfigDefaults.reporter),
lines: config['100'] ? 100 : config.lines,
functions: config['100'] ? 100 : config.functions,
branches: config['100'] ? 100 : config.branches,
statements: config['100'] ? 100 : config.statements,

thresholds: config.thresholds && {
...config.thresholds,
lines: config.thresholds['100'] ? 100 : config.thresholds.lines,
branches: config.thresholds['100'] ? 100 : config.thresholds.branches,
functions: config.thresholds['100'] ? 100 : config.thresholds.functions,
statements: config.thresholds['100'] ? 100 : config.thresholds.statements,
},
}

this.instrumenter = createInstrumenter({
Expand Down Expand Up @@ -170,34 +175,36 @@ export class IstanbulCoverageProvider extends BaseCoverageProvider implements Co
}).execute(context)
}

if (this.options.branches
|| this.options.functions
|| this.options.lines
|| this.options.statements) {
this.checkThresholds({
if (this.options.thresholds) {
const resolvedThresholds = this.resolveThresholds({
coverageMap,
thresholds: {
branches: this.options.branches,
functions: this.options.functions,
lines: this.options.lines,
statements: this.options.statements,
},
perFile: this.options.perFile,
thresholds: this.options.thresholds,
createCoverageMap: () => libCoverage.createCoverageMap({}),
})
}

if (this.options.thresholdAutoUpdate && allTestsRun) {
this.updateThresholds({
coverageMap,
thresholds: {
branches: this.options.branches,
functions: this.options.functions,
lines: this.options.lines,
statements: this.options.statements,
},
perFile: this.options.perFile,
configurationFile: this.ctx.server.config.configFile,
this.checkThresholds({
thresholds: resolvedThresholds,
perFile: this.options.thresholds.perFile,
})

if (this.options.thresholds.autoUpdate && allTestsRun) {
if (!this.ctx.server.config.configFile)
throw new Error('Missing configurationFile. The "coverage.thresholds.autoUpdate" can only be enabled when configuration file is used.')

const configFilePath = this.ctx.server.config.configFile
const configModule = parseModule(await fs.readFile(configFilePath, 'utf8'))

this.updateThresholds({
thresholds: resolvedThresholds,
perFile: this.options.thresholds.perFile,
configurationFile: {
write: () => writeFileSync(configFilePath, configModule.generate().code, 'utf-8'),
read: () => configModule.exports.default.$type === 'function-call'
? configModule.exports.default.$args[0]
: configModule.exports.default,
},
})
}
}
}

Expand Down
1 change: 1 addition & 0 deletions packages/coverage-v8/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@
"istanbul-lib-source-maps": "^4.0.1",
"istanbul-reports": "^3.1.6",
"magic-string": "^0.30.5",
"magicast": "^0.3.2",
"picocolors": "^1.0.0",
"std-env": "^3.4.3",
"test-exclude": "^6.0.0",
Expand Down
65 changes: 36 additions & 29 deletions packages/coverage-v8/src/provider.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { existsSync, promises as fs } from 'node:fs'
import { existsSync, promises as fs, writeFileSync } from 'node:fs'
import type { Profiler } from 'node:inspector'
import { fileURLToPath, pathToFileURL } from 'node:url'
import v8ToIstanbul from 'v8-to-istanbul'
Expand All @@ -9,6 +9,7 @@ import type { CoverageMap, CoverageMapData } from 'istanbul-lib-coverage'
import libCoverage from 'istanbul-lib-coverage'
import libSourceMaps from 'istanbul-lib-source-maps'
import MagicString from 'magic-string'
import { parseModule } from 'magicast'
import remapping from '@ampproject/remapping'
import { normalize, resolve } from 'pathe'
import c from 'picocolors'
Expand Down Expand Up @@ -72,10 +73,14 @@ export class V8CoverageProvider extends BaseCoverageProvider implements Coverage
provider: 'v8',
reporter: this.resolveReporters(config.reporter || coverageConfigDefaults.reporter),
reportsDirectory: resolve(ctx.config.root, config.reportsDirectory || coverageConfigDefaults.reportsDirectory),
lines: config['100'] ? 100 : config.lines,
functions: config['100'] ? 100 : config.functions,
branches: config['100'] ? 100 : config.branches,
statements: config['100'] ? 100 : config.statements,

thresholds: config.thresholds && {
...config.thresholds,
lines: config.thresholds['100'] ? 100 : config.thresholds.lines,
branches: config.thresholds['100'] ? 100 : config.thresholds.branches,
functions: config.thresholds['100'] ? 100 : config.thresholds.functions,
statements: config.thresholds['100'] ? 100 : config.thresholds.statements,
},
}

this.testExclude = new _TestExclude({
Expand Down Expand Up @@ -156,34 +161,36 @@ export class V8CoverageProvider extends BaseCoverageProvider implements Coverage
}).execute(context)
}

if (this.options.branches
|| this.options.functions
|| this.options.lines
|| this.options.statements) {
this.checkThresholds({
if (this.options.thresholds) {
const resolvedThresholds = this.resolveThresholds({
coverageMap,
thresholds: {
branches: this.options.branches,
functions: this.options.functions,
lines: this.options.lines,
statements: this.options.statements,
},
perFile: this.options.perFile,
thresholds: this.options.thresholds,
createCoverageMap: () => libCoverage.createCoverageMap({}),
})
}

if (this.options.thresholdAutoUpdate && allTestsRun) {
this.updateThresholds({
coverageMap,
thresholds: {
branches: this.options.branches,
functions: this.options.functions,
lines: this.options.lines,
statements: this.options.statements,
},
perFile: this.options.perFile,
configurationFile: this.ctx.server.config.configFile,
this.checkThresholds({
thresholds: resolvedThresholds,
perFile: this.options.thresholds.perFile,
})

if (this.options.thresholds.autoUpdate && allTestsRun) {
if (!this.ctx.server.config.configFile)
throw new Error('Missing configurationFile. The "coverage.thresholds.autoUpdate" can only be enabled when configuration file is used.')

const configFilePath = this.ctx.server.config.configFile
const configModule = parseModule(await fs.readFile(configFilePath, 'utf8'))

this.updateThresholds({
thresholds: resolvedThresholds,
perFile: this.options.thresholds.perFile,
configurationFile: {
write: () => writeFileSync(configFilePath, configModule.generate().code, 'utf-8'),
read: () => configModule.exports.default.$type === 'function-call'
? configModule.exports.default.$args[0]
: configModule.exports.default,
},
})
}
}
}

Expand Down
2 changes: 1 addition & 1 deletion packages/vitest/src/node/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ cli
.option('--hideSkippedTests', 'Hide logs for skipped tests')
.option('--reporter <name>', 'Specify reporters')
.option('--outputFile <filename/-s>', 'Write test results to a file when supporter reporter is also specified, use cac\'s dot notation for individual outputs of multiple reporters')
.option('--coverage', 'Enable coverage report', { default: { 100: false } })
.option('--coverage', 'Enable coverage report')
.option('--run', 'Disable watch mode')
.option('--mode <name>', 'Override Vite mode (default: test)')
.option('--globals', 'Inject apis globally')
Expand Down
Loading