-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migrate E2E tests to Playwright #38851
Comments
cc @kevin940726 |
I reformatted this issue a little bit, added some links, and un-checked all the checkboxes since, to my knowledge, we haven't merged anything that migrates any tests yet. (#38570 is just the infrastructure.) |
It would be nice to flag 🚩 which tests are flaky so that we can prioritise them. That will show us if Playwright offers any improvements there. |
I removed the e2e-utils checkboxes from the issue description, since they will be added if necessary when we are migrating tests. |
@JustinyAhin Can you please update the https://github.com/WordPress/gutenberg/blob/trunk/test/e2e/MIGRATION.md |
@juhi123 Updated! Thanks! |
I started looking at the second most flakey test ( I hit a roadblock that I think is worth trying to solve early on while there's a chance to put the right structure in place. The site editor is using an iframe for the editor canvas. Because of this, none of the existing test utils for interacting with the editor canvas can be used for site editor tests. That's because they all use It's clear this was also a problem in Puppeteer, most of the puppeteer site editor tests have duplicate utility functions in the test file. I'm not all that familiar with playwright, but I wonder if it could be an option to move the canvas utils into a class and do something like the following: test.use( {
editorCanvasUtils: async (
{ page, pageUtils },
use
) => {
await use(
new EditorCanvasUtils( { page, pageUtils, isSiteEditor: true } )
);
},
} );
|
Every end-to-end test has been migrated to the Playwright. Thank you everyone for contributing! |
Thank you all! ❤️ 💯 There is still some work left to do to clean up. We'll do them in follow-up PRs! |
This issue tracks progress on migrating Gutenberg's E2E tests from Puppeteer (
packages/e2e-tests
) to Playwright (test/e2e
).Resources:
Tests to migrate
Flaky tests
These tests are ranked by their estimated flaky rates. Experience developers should prioritize migrating them in this order.
editor/blocks/image.test.js
(Migrate and refactor image block e2e tests to Playwright #40804)site-editor/template-part.test.js
(Migrate template part test to playwright #41048)editor/various/post-editor-template-mode.test.js
(Migratepost-editor-template-mode
to Playwright #40175)site-editor/template-revert.test.js
(Migrate revert template test to Playwright #41310)site-editor/document-settings.test.js
(Migrate document settings test to playwright #40258)editor/various/preview.test.js
(Refactor preview e2e tests to use Playwright. #41380)editor/blocks/navigation.test.js
(Navigation block e2e tests: default to a list of pages if there are no menus #45070)editor/blocks/gallery.test.js
(Migrate Gallery tests to Playwright #45202)editor/plugins/templates.test.js
(Migratetemplates
e2e test to Playwright #45393)editor/various/list-view.test.js
(Migrate list view E2E tests to Playwright #47919)editor/plugins/iframed-multiple-block-stylesheets.test.js
( Migrate 'iframed multiple block stylesheets' tests to Playwright #55003) (Migrate iframed multiple block stylesheet test to playwright #45622)site-editor/style-variations.test.js
(E2E Tests: Migrate style variations tests to Playwright #41427)site-editor/settings-sidebar.test.js
(Migrate/setting sidebar #46174) (Migrate Site Editor 'settings sidebar' e2e tests to Playwright #57392)editor/various/autosave.test.js
(Migrate autosave E2E tests to Playwright #49026) (Migrate 'autosave' e2e tests to Playwright #58171)experiments/blocks/comments-query.test.js
(E2E Tests: Migrate Comments block test to Playwright #39826)editor/various/writing-flow.test.js
(Migrate writing flow tests to Playwright #43637)editor/various/taxonomies.test.js
(Migrate 'taxonomies' e2e tests to Playwright #57662)editor/various/block-grouping.test.js
(Migrate 'block grouping' e2e tests to Playwright #57684)editor/various/change-detection.test.js
(Migratechange-detection
to Playwright #58767)editor/various/publish-button.test.js
(Migrate flaky PostPublishButton e2e tests to Playwright #52285)editor/various/toolbar-roving-tabindex.test.js
(Migrate Toolbar roving tabindex tests to Playwright #46634)widgets/editing-widgets.test.js
(Migrateediting-widgets
to Playwright #57483)editor/blocks/list.test.js
(Migrate list block test to Playwright #41555)editor/various/embedding.test.js
(Migrate 'embedding' e2e tests to Playwright #57969)editor/various/inserting-blocks.test.js
(Migrate remaining 'inserting blocks' e2e tests to Playwright #58108)editor/various/block-editor-keyboard-shortcuts.test.js
(Migrate 'block editor keyboard shortcuts' e2e tests to Playwright #57422)editor/various/multi-block-selection.test.js
(Migrate multi-block selection E2E tests to Playwright #48035)editor/plugins/custom-post-types.test.js
(Migrate CPT e2e tests to Playwright #50031)editor/various/block-deletion.test.js
(Migrate block deletion E2E tests to Playwright #48012)editor/blocks/classic.test.js
(Migrate Classic block tests to Playwright #46689)editor/blocks/post-title.test.js
(Migrate 'Post Title block' e2e tests to Playwright #55297)editor/blocks/group.test.js
(Migrate group block tests to Playwright #42801)editor/blocks/site-title.test.js
(Migrate 'Site Title block' e2e tests to Playwright #55704)editor/plugins/iframed-inline-styles.test.js
(Migrate iframed inline-styles test to playwright #44498) (Migrate 'iframed inline styles' tests to Playwright #55009)editor/plugins/meta-boxes.test.js
(Migrate wp editor meta box test to Playwright #41519)editor/various/copy-cut-paste-whole-blocks.test.js
(Migrate cut-copy-paste-whole-blocks to Playwright #39807)editor/various/draggable-block.test.js
(Migrate draggable block tests to Playwright #43798)editor/various/keyboard-navigable-blocks.test.js
(Migrate keyboard-navigable-blocks e2e tests from puppeteer to playwright #54944)editor/various/links.test.js
(Link control: migrate tests to Playwright. Can be created by selecting text and using keyboard shortcuts #50996)editor/various/manage-reusable-blocks.test.js
(Migrate Manage Reusable Block to Playwright #45621)editor/various/rich-text.test.js
(Migrate RichText e2e tests to Playwright #53493)editor/various/undo.test.js
(Migrate undo to Playwright #48701)site-editor/multi-entity-saving.test.js
(Migrate site editor multi-entity save flow tests #52372)site-editor/site-editor-export.test.js
(Migrate 'site editor export' e2e tests to Playwright #56675)widgets/customizing-widgets.test.js
(Migrate customizing-widgets to Playwright #39540)Other tests
These tests don't have corresponding flaky reports. Newcomers can start migrating these since they are more stable and probably easier to start with.
editor/blocks/buttons.test.js
(Migrate Button Block tests to Playwright #41494)editor/blocks/code.test.js
(Migrate code block tests to Playwright #41136)editor/blocks/columns.test.js
(Migrate columns test case to Playwright #43964)editor/blocks/cover.test.js
(Migrate Cover Block tests to Playwright #45784)editor/blocks/heading.test.js
(Migrate Heading block tests to Playwright #47955)editor/blocks/html.test.js
(Migrate html block test case to Playwright #41231)editor/blocks/missing.test.js
(Migrate missing block tests to Playwright #41680)editor/blocks/paragraph.test.js
(Migrate paragraph test to Playwright #40779)editor/blocks/preformatted.test.js
(Migrate preformatted block tests to Playwright #41128)editor/blocks/query.test.js
(Migrate query test to Playwright #47995)editor/blocks/quote.test.js
(Migrate quote e2e tests to playwright #46549)editor/blocks/separator.test.js
(Migrate separator block tests to Playwright #41130)editor/blocks/spacer.test.js
(Migrate spacer test to playwright #41590)editor/blocks/table.test.js
(Migrate table block tests to Playwright #41945)editor/blocks/pullquote.test.js
(Migrate PullQuote test case to playwright #50085)editor/plugins/align-hook.test.js
(Migrate 'align hook' e2e tests to Playwright #56480)editor/plugins/allowed-blocks.test.js
(Migrate Allowed Block Test to Playwright #53171)editor/plugins/annotations.test.js
(Migrate 'annotations' e2e tests to Playwright #55966)editor/plugins/block-api.test.js
(Migrate block api test to Playwright #40217)editor/plugins/block-context.test.js
(Migrate 'block context' e2e tests to Playwright #55793)editor/plugins/block-directory-add.test.js
(Migrate 'block directory' e2e tests to Playwright #56593)editor/plugins/block-icons.test.js
(WIP Add/playwright block icon #55501) (Migrate 'block icons' e2e tests to Playwright #56610)editor/plugins/block-variations.test.js
(Migrate 'Block variations' e2e tests to Playwright #53266)editor/plugins/child-blocks.test.js
(Migrate Child Block Test to Playwright #55199)editor/plugins/container-blocks.test.js
(Migrate 'container blocks' e2e tests to Playwright #56141)editor/plugins/cpt-locking.test.js
(Migrate 'CPT locking' e2e tests to Playwright #55929)editor/plugins/custom-taxonomies.test.js
(Migrate custom taxonomies test to Playwright #45178) (Migrate 'custom taxonomies' e2e test to Playwright #56486)editor/plugins/deprecated-node-matcher.test.js
(Migrate deprecated node matcher tests to playwright #42759)editor/plugins/format-api.test.js
(Migrate format API test for playwright #42035)editor/plugins/hooks-api.test.js
(Migrate hooks api test to playwright #42584)editor/plugins/iframed-block.test.js
(Migrate iframed block test to Playwright #44164)editor/plugins/iframed-masonry-block.test.js
(#Migrate 'iframed masonry block' tests to Playwright #55016)editor/plugins/image-size.test.js
(Migrate Image Size to Playwright #40467)editor/plugins/inner-blocks-allowed-blocks.test.js
(Migrate 'Allowed Blocks Setting on InnerBlocks' tests to Playwright #51677)editor/plugins/inner-blocks-render-appender.test.js
(Migrate 'inner-blocks-render-appender' e2e tests to Playwright #55814)editor/plugins/meta-attribute-block.test.js
(#Migrate 'meta-attribute-block' e2e tests to Playwright #55830)editor/plugins/nonce.test.js
(Migratenonce.test.js
to Playwright #44929)editor/plugins/plugins-api.test.js
(Migrate 'Plugins API' e2e tests to Playwright #55958)editor/plugins/register-block-type-hooks.test.js
(Migrate register block test case to playwright #43170)editor/plugins/wp-editor-meta-box.test.js
(Migrate wp editor meta box test to Playwright #41519)editor/various/a11y.test.js
(Migrate a11y test to Playwright #39638) (Migrate a11y e2e tests to Playwright #46038)editor/various/adding-inline-tokens.test.js
(Migrate Adding Inline Token test to Playwright #52020)editor/various/adding-patterns.test.js
(Migrate Adding Patterns Test to Playwright #50083)editor/various/allowed-patterns.test.js
(Migrate 'allowed patterns' e2e tests to Playwright #57399)editor/various/block-hierarchy-navigation.test.js
(Migrate block hierarchy navigation tests to Playwright #51517)editor/various/block-locking.test.js
(Block Locking: Fix regression in selectors #51541)editor/various/block-mover.test.js
(Migrate Block Mover Test For Playwright #42039)editor/various/block-switcher.test.js
(Migrate Block Switcher Test case to Playwright #50845) (Migrate remaining 'block switcher' e2e tests to Playwright #57381)editor/various/compatibility-classic-editor.test.js
(Add Playwright Compatibility-classic-editor Test #43979)editor/various/convert-block-type.test.js
(Migrate Convert Block Type test to Playwright #42760)editor/various/core-settings.test.js
(Migrate core-settings #46176) (Migrate 'core settings' e2e tests to Playwright #57581)editor/various/datepicker.test.js
(Migrate 'datepicker' e2e tests to Playwright #57545)editor/various/dropdown-menu.test.js
(Migrate 'dropdown menu' e2e tests to Playwright #57663)editor/various/duplicating-blocks.test.js
(Migrate duplicating block test to playwright #43171)editor/various/editor-modes.test.js
(Migrate 'editor modes' e2e tests to Playwright #57574)editor/various/font-size-picker.test.js
(Migrate Font Size Picker tests to Playwright #46591)editor/various/format-library/text-color.test.js
(Migrate 'text color' e2e test to Playwright #55323)editor/various/fullscreen-mode.test.js
(Migrate Fullscreen mode test to Playwright #43963)editor/various/invalid-block.test.js
(Migrate 'invalid blocks' e2e tests to Playwright #57508)editor/various/is-typing.test.js
(Migrate Is Typing Test to Playwright #56616)editor/various/keep-styles-on-block-transforms.test.js
(Migrate Keep Transform Block Test Case to Playwright #49719)editor/various/manage-reusable-blocks.test.js
(Migrate Manage Reusable Block to Playwright #45621)editor/various/mentions.test.js
(Migrate mentions tests to playwright #43064)editor/various/navigable-toolbar.test.js
(Migrate Navigable toolbar test to Playwright #51514)editor/various/new-post-default-content.test.js
(Migrate New Default Post Content Test to Playwright #45267)editor/various/new-post.test.js
(Migrate new-post to Playwright #39539)editor/various/nux.test.js
(Migrated nux.test.js to Playwright #39772) (Migrate 'nux' e2e tests to Playwright #57542)editor/various/pattern-blocks.test.js
(Migrate remaining 'pattern block' e2e tests to Playwright #58486)editor/various/popovers.test.js
(Migrated popovers.test.js to Playwright #39910)editor/various/post-visibility.test.js
(Migrate Post Visibility Test to Playwright #44488)editor/various/preferences.test.js
(Migrate 'preferences' e2e tests to Playwright #57446)editor/various/publish-panel.test.js
(#Migrate remaining 'publish panel' e2e tests to Playwright #57432)editor/various/publishing.test.js
(Migrate 'publishing' e2e tests to Playwright #57521)editor/various/rtl.test.js
(Migrate rtl test case to it's playwright version #41495)editor/various/scheduling.test.js
(Migrate 'scheduling' e2e tests to Playwright #57539)editor/various/shortcut-help.test.js
(Migrate shortcut help e2e tests to Playwright #53832)editor/various/sidebar-permalink.test.js
(Migrate 'sidebar permalink' e2e tests to Playwright #56253)editor/various/sidebar.test.js
(Migrate 'sidebar' e2e tests to Playwright #57448)editor/various/splitting-merging.test.js
(Migrate Splitting Merging test to Playwright #44916)editor/various/style-variation.test.js
(Migrate style variation test to Playwright #40216)editor/various/switch-to-draft.test.js
(Migrateswitch-to-draft
to Playwright #48120)editor/various/typewriter.test.js
(Migrate 'typewriter' e2e tests to Playwright #57673)local/demo.test.js
(Migrate demo page tests to Playwright #55054)performance/post-editor.test.js
(Migrate performance tests to Playwright #51084) (Switch performance tests to Playwright #52022)performance/site-editor.test.js
(Migrate performance tests to Playwright #51084) (Switch performance tests to Playwright #52022)site-editor/iframe-rendering-mode.test.js
(Migrate iframe-rendering test case #44535)site-editor/multi-entity-saving.test.js
(Migrate site editor multi-entity save flow tests #52372) (Migrate 'editor multi entity saving' e2e tests to Playwright #56670)site-editor/site-editor-inserter.test.js
(Migratesite-editor-inserter
e2e test to Playwright #44507)Performance tests
packages/e2e-tests/specs/performance
(Migrate performance tests to Playwright #51084, Switch performance tests to Playwright #52022)The text was updated successfully, but these errors were encountered: