Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[*] Chore: Mark additional tests as flaky from #6535 test runs #6536

Merged
merged 2 commits into from
Aug 21, 2024

Conversation

etrepum
Copy link
Collaborator

@etrepum etrepum commented Aug 21, 2024

Description

We still have a lot of tests that are flaky, which is a considerable burden on contributions and reviewing them. This is just the ones that flaked in #6535 on this particular run https://github.com/facebook/lexical/actions/runs/10481362302/job/29033662935

Test plan

Before

 9 flaky
    [chromium] › packages/lexical-playground/__tests__/e2e/CopyAndPaste/lexical/ListsCopyAndPaste.spec.mjs:111:3 › Lists CopyAndPaste › Copy and paste of partial list items into the list 
    [chromium] › packages/lexical-playground/__tests__/e2e/HorizontalRule.spec.mjs:30:3 › HorizontalRule › Can create a horizontal rule and move selection around it 
    [chromium] › packages/lexical-playground/__tests__/e2e/Links.spec.mjs:1926:3 › Links › Can handle pressing Enter inside a Link 
    [chromium] › packages/lexical-playground/__tests__/e2e/Links.spec.mjs:1960:3 › Links › Can handle pressing Enter inside a Link containing multiple TextNodes 
    [chromium] › packages/lexical-playground/__tests__/e2e/Selection.spec.mjs:877:3 › Selection › shift+arrowdown into a table, when the table is the last node, selects the whole table 
    [chromium] › packages/lexical-playground/__tests__/e2e/Selection.spec.mjs:927:3 › Selection › shift+arrowdown into a table, when the table is the only node, selects the whole table 
    [chromium] › packages/lexical-playground/__tests__/e2e/Tab.spec.mjs:20:3 › Tab › can tab + IME ─
    [chromium] › packages/lexical-playground/__tests__/e2e/TextFormatting.spec.mjs:1114:3 › TextFormatting › Regression #2523: can toggle format when selecting a TextNode edge followed by a non TextNode;  
    [chromium] › packages/lexical-playground/__tests__/regression/429-swapping-emoji.spec.mjs:21:3 › Regression test #429 › Can add new lines before the line with emoji 

After

These 9 tests are tagged @flaky

Copy link

vercel bot commented Aug 21, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
lexical ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 21, 2024 1:45pm
lexical-playground ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 21, 2024 1:45pm

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 21, 2024
@etrepum etrepum added the extended-tests Run extended e2e tests on a PR label Aug 21, 2024
Copy link

github-actions bot commented Aug 21, 2024

size-limit report 📦

Path Size
lexical - cjs 29.38 KB (0%)
lexical - esm 29.24 KB (0%)
@lexical/rich-text - cjs 37.74 KB (0%)
@lexical/rich-text - esm 31.05 KB (0%)
@lexical/plain-text - cjs 36.41 KB (0%)
@lexical/plain-text - esm 28.42 KB (0%)
@lexical/react - cjs 39.64 KB (0%)
@lexical/react - esm 32.51 KB (0%)

@etrepum
Copy link
Collaborator Author

etrepum commented Aug 21, 2024

The plain text tests are expected to fail until #6535 is merged and this is updated

@potatowagon
Copy link
Contributor

the criteria to mark flaky (decided with @Sahejkm ) was if a test failed on a PR due to flakiness. however also ok to adjust the criteria to occasionally include non failing tests with flaky runs, since it was a bother

@etrepum
Copy link
Collaborator Author

etrepum commented Aug 21, 2024

Every test that has a flaky run has a chance of failing, it's just probability. The more we have the more often it's going to be a problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. extended-tests Run extended e2e tests on a PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants