Coverage Report Path in Browser Mode Does Not Consider "reportsDirectory"

[oscarmarina]

Describe the bug

Hello,

I am experiencing an issue with the coverage report in Vitest when using Browser Mode with a web component built with Lit. When I run the tests and try to view the coverage, the page is blank. It seems that the coverage report is looking for the folder at the root of the project, not taking into account the reportsDirectory setting in my vite.config.js file.

reportsDirectory:./test/coverage/,

==

I would like to suggest an improvement for the LCOV reporter. Currently, the LCOV reporter generates the same files as the HTML reporter. However, I believe the LCOV reporter could be improved by creating a separate folder for its output. This would keep everything more organized, instead of leaving all the files in the root of the /coverage folder. Furthermore, when the Vitest UI is activated, the LCOV reporter is "equated" to the HTML reporter for generating the report view.

Reproduction

You can reproduce this issue by downloading the repository from my GitHub or viewing it on Stackblitz.

https://github.com/oscarmarina/lit-vitest-testing-comparison

https://stackblitz.com/github/oscarmarina/lit-vitest-testing-comparison

npm i && npm run test

System Info

System:
    OS: macOS 12.7.2
    CPU: (4) x64 Intel(R) Core(TM) i5-5287U CPU @ 2.90GHz
    Memory: 17.82 MB / 16.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 20.10.0 - ~/.nvm/versions/node/v20.10.0/bin/node
    Yarn: 1.22.19 - ~/.nvm/versions/node/v20.10.0/bin/yarn
    npm: 10.3.0 - ~/.nvm/versions/node/v20.10.0/bin/npm
    pnpm: 8.6.5 - ~/.nvm/versions/node/v20.10.0/bin/pnpm
  Browsers:
    Chrome Canary: 122.0.6257.0
    Safari: 17.2.1

Used Package Manager

npm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[AriPerkkio]

Has this been working with earlier versions of Vitest?

I think this part of the UI is the root cause. For some reason the reportsDirectory is sliced from the last /, so your ./test/coverage becomes just /coverage.

vitest/packages/ui/client/composables/navigation.ts

Lines 15 to 30 in 7314857

	// TODO
	// For html report preview, "coverage.reportsDirectory" must be explicitly set as a subdirectory of html report.
	// Handling other cases seems difficult, so this limitation is mentioned in the documentation for now.
	export const coverageUrl = computed(() => {
	if (coverageEnabled.value) {
	const idx = coverage.value!.reportsDirectory.lastIndexOf('/')
	const htmlReporter = coverage.value!.reporter.find((reporter) => {
	if (reporter[0] !== 'html')
	return undefined
	return reporter
	})
	return htmlReporter && 'subdir' in htmlReporter[1]
	? `/${coverage.value!.reportsDirectory.slice(idx + 1)}/${htmlReporter[1].subdir}/index.html`
	: `/${coverage.value!.reportsDirectory.slice(idx + 1)}/index.html`
	}

I would like to suggest an improvement for the LCOV reporter. Currently, the LCOV reporter generates the same files as the HTML reporter.

These built-in reporters are provided by Istanbul. Changes for them should be requested at https://github.com/istanbuljs/istanbuljs.

But it sounds like you should rather use lcovonly reporter if you have html reporter enabled:

lcov

Same as lcovonly, but also generates a HTML report you can view in your browser

https://istanbul.js.org/docs/advanced/alternative-reporters/#lcov

[oscarmarina]

Hi @AriPerkkio, this is the first time I use vitest, I don't know if it worked before, sorry.

I explained poorly about lcov, what I was suggesting is to be able to use lcov to activate the coverage tab

[hi-ogawa]

I think currently there's a difference between "vitest:browser" middleware plugin and "vitest:ui" plugin. "vitest:browser" only serves its own html/js assets but "vitest:ui" also serves generated coverage html:

vitest/packages/browser/src/node/index.ts

Lines 25 to 28 in d53b858

	async configureServer(server) {
	server.middlewares.use(
	base,
	sirv(resolve(distRoot, 'client'), {

vitest/packages/ui/node/index.ts

Line 20 in d53b858

coverageFolder && server.middlewares.use(coveragePath!, sirv(coverageFolder[0], {

This "server side" code might also need to be adjusted for coverage preview on (headed) browser mode.

[oscarmarina]

Hi, I've set up a branch using provider: 'playwright', just as a helpful reference.

• https://github.com/oscarmarina/lit-vitest-testing-comparison/tree/feature/playwright

[userquin]

On my Windows laptop I can see this weird error when running the reproduction:

[userquin]

The problem seems to be here: https://github.com/oscarmarina/lit-vitest-testing-comparison/blob/main/index.html#L8

If I change previous line using absolute path /demo/index.html, the coverage folder showing the example:

[userquin]

Vitest getting this error:

[userquin]

@vitest/ui plugin not being included since ui is false and browser package not including coverage logic, so #5056 seems to be fine.

[oscarmarina]

hi @userquin
the istanbul HTML report looks like this:

====
When you remove this line the "coverage" folder is created in the root of the project that's when it works fine

[userquin]

It works (when removing the directory option) because it is on the root, once you include the report directory the client will request /coverage/index.html and since there is no middleware to serve /coverage/ requests from the test/coverage directory, it fails, there is no coverage folder in root.

IIRC I did the original PR to include coverage in the ui 😅