Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(test): ensure screenshot dir is cleaned up #5421

Merged
merged 2 commits into from
Mar 6, 2024

Conversation

christian-bromann
Copy link
Member

What is the current behavior?

Currently we don't clean our screenshot baseline which causes issues when run locally as captured screenshot don't match.

STENCIL-1095

What is the new behavior?

Remove the baseline as part of the clean up process. It seems we never kept the baseline as part of the repository which means that we never actually verified the end result of the screenshot tests. The screenshot tests we run therefore only verify that the command can be called and the screenshot can be stored on the system. However we don't verify whether consecutive tests keep matching with the baseline.

I evaluated two options:

  • actually check in the baseline (test/end-to-end/screenshot) and give these tests more value or
  • remove the baseline regularly so that Stencil developers don't run into a failing test

Given that the screenshot comparison feature is still (and possibly remains) experimental and we haven't planned to work on this, I decided for the latter.

Documentation

Does this introduce a breaking change?

  • Yes
  • No

Testing

Run npm run test.end-to-end and everything should pass.

Other information

n/a

Copy link
Contributor

--strictNullChecks error report

Typechecking with --strictNullChecks resulted in 1139 errors on this branch.

That's the same number of errors on main, so at least we're not creating new ones!

reports and statistics

Our most error-prone files
Path Error Count
src/dev-server/index.ts 37
src/dev-server/server-process.ts 32
src/compiler/prerender/prerender-main.ts 22
src/testing/puppeteer/puppeteer-element.ts 22
src/runtime/client-hydrate.ts 20
src/screenshot/connector-base.ts 19
src/runtime/vdom/vdom-render.ts 17
src/dev-server/request-handler.ts 15
src/compiler/prerender/prerender-optimize.ts 14
src/compiler/sys/stencil-sys.ts 14
src/compiler/transpile/transpile-module.ts 14
src/sys/node/node-sys.ts 14
src/compiler/prerender/prerender-queue.ts 13
src/compiler/sys/in-memory-fs.ts 13
src/runtime/connected-callback.ts 13
src/runtime/set-value.ts 13
src/compiler/output-targets/output-www.ts 12
src/compiler/transformers/test/parse-vdom.spec.ts 12
src/compiler/transformers/transform-utils.ts 12
src/mock-doc/test/attribute.spec.ts 12
Our most common errors
Typescript Error Code Count
TS2322 361
TS2345 349
TS18048 201
TS18047 82
TS2722 37
TS2532 24
TS2531 21
TS2454 14
TS2790 11
TS2352 10
TS2769 8
TS2538 8
TS2416 6
TS2493 3
TS18046 2
TS2684 1
TS2430 1

Unused exports report

There are 14 unused exports on this PR. That's the same number of errors on main, so at least we're not creating new ones!

Unused exports
File Line Identifier
src/runtime/bootstrap-lazy.ts 21 setNonce
src/screenshot/screenshot-fs.ts 18 readScreenshotData
src/testing/testing-utils.ts 198 withSilentWarn
src/utils/index.ts 145 CUSTOM
src/utils/index.ts 269 normalize
src/utils/index.ts 7 escapeRegExpSpecialCharacters
src/compiler/app-core/app-data.ts 25 BUILD
src/compiler/app-core/app-data.ts 115 Env
src/compiler/app-core/app-data.ts 117 NAMESPACE
src/compiler/fs-watch/fs-watch-rebuild.ts 123 updateCacheFromRebuild
src/compiler/types/validate-primary-package-output-target.ts 61 satisfies
src/compiler/types/validate-primary-package-output-target.ts 61 Record
src/testing/puppeteer/puppeteer-declarations.ts 485 WaitForEventOptions
src/compiler/sys/fetch/write-fetch-success.ts 7 writeFetchSuccessSync

Copy link
Contributor

github-actions bot commented Feb 28, 2024

PR built and packed!

Download the tarball here: https://github.com/ionic-team/stencil/actions/runs/8088639302/artifacts/1284222179

If your browser saves files to ~/Downloads you can install it like so:

unzip -d ~/Downloads ~/Downloads/stencil-core-4.12.4-dev.1709163931.901fcc4.tgz.zip && npm install ~/Downloads/stencil-core-4.12.4-dev.1709163931.901fcc4.tgz

@christian-bromann christian-bromann added this pull request to the merge queue Mar 6, 2024
Merged via the queue into main with commit 15e7a49 Mar 6, 2024
121 checks passed
@christian-bromann christian-bromann deleted the cb/clean-screenshots-dir branch March 6, 2024 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants