Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GBonWPCOM e2e tests i0] Add assertion for blocks in the frontend (published) page for edge/non-edge, add @parallel tag #45794

Merged
merged 32 commits into from
Oct 2, 2020

Conversation

fullofcaffeine
Copy link
Contributor

@fullofcaffeine fullofcaffeine commented Sep 22, 2020

Changes proposed in this Pull Request

This is a follow up for #45048. Tracking issue: #45209.

  • The @parallel tag (one per block/edge+non edge site test)has been added so that the test is picked up by magellan/CI and will run faster (< 6min timeout limit);
  • Initial groundwork that checks for the blocks that should be rendered; this means adding a selector to the block classes for the blocks that already render on the frontend.
  • Some blocks require additional setup before appear on the frontend and are not included as part of this changeset (see other related PRs).
  • Fixes an issue that was preventing screenshots from being created on the CI VM.

This was extracted from the PR here: #45284. We decided to split it into several PRs, instead. The amount of files changed is the same (due to the static property being added to the block classes with the frontend selector) but the actual changeset is simpler, as the logic for filling/setting-up each block has been extracted into other PRs.

⚠️ The goal is to review and merge this first so that the other related PR's (see below) diffs become simpler and are easier to review and merge after this one ⚠️

Why

The initial task added the framework/PoC for automating our manual testing process. It only configured two blocks, though. The goal of this PR is to lay the groundwork to later fill in all of the popular blocks being tested as part of the upgrade E2E test, and verify them in the published page too (edge/non-edge and across the most popular themes).

Testing instructions

  1. Setup your env to run Calypso E2E tests (search for "Automated end-to-end Testing") in the FG;
  2. Run with ./node_modules/.bin/mocha specs/wp-calypso-gutenberg-upgrade-spec.js*[0]
  3. Wait for the tests to finish. It might take a good while since it tests everything for each test site
  4. Make sure it didn't err, if it did, it might not be an actual failure, try running that specific step again. If it continues failing, let us know
  5. Add the Needs e2e Testing Gutenberg Edge to this PR (already added, if necessary, remove and add again);
  6. Check this branch's tests in CI: Desktop, Mobile.

*[0] Make sure to cd into test/e2e first.

Related PRs

Review and test the following PRs after this PR has been merged:

@matticbot
Copy link
Contributor

@matticbot
Copy link
Contributor

This PR does not affect the size of JS and CSS bundles shipped to the user's browser.

Generated by performance advisor bot at iscalypsofastyet.com.

@sirreal sirreal added [Status] Needs e2e Testing Gutenberg Edge Run e2e tests with a special site that is running the latest Gutenberg plugin snapshot and removed [Status] Needs e2e Testing Gutenberg Edge Run e2e tests with a special site that is running the latest Gutenberg plugin snapshot labels Sep 22, 2020
@fullofcaffeine fullofcaffeine added [Status] Needs e2e Testing Gutenberg Edge Run e2e tests with a special site that is running the latest Gutenberg plugin snapshot and removed [Status] Needs e2e Testing Gutenberg Edge Run e2e tests with a special site that is running the latest Gutenberg plugin snapshot labels Sep 23, 2020
@fullofcaffeine fullofcaffeine self-assigned this Sep 23, 2020
@WunderBart WunderBart added [Status] Needs e2e Testing Gutenberg Edge Run e2e tests with a special site that is running the latest Gutenberg plugin snapshot and removed [Status] Needs e2e Testing Gutenberg Edge Run e2e tests with a special site that is running the latest Gutenberg plugin snapshot labels Sep 23, 2020
@fullofcaffeine fullofcaffeine force-pushed the add/gb-upgrade-fill-in-blocks-groundwork branch 2 times, most recently from 7da339a to 39dbff5 Compare September 23, 2020 21:36
@fullofcaffeine fullofcaffeine added [Status] Needs e2e Testing Gutenberg Edge Run e2e tests with a special site that is running the latest Gutenberg plugin snapshot and removed [Status] Needs e2e Testing Gutenberg Edge Run e2e tests with a special site that is running the latest Gutenberg plugin snapshot labels Sep 23, 2020
@fullofcaffeine fullofcaffeine force-pushed the add/gb-upgrade-fill-in-blocks-groundwork branch from 39dbff5 to 8500487 Compare September 24, 2020 00:15
@WunderBart WunderBart added [Status] Needs e2e Testing Gutenberg Edge Run e2e tests with a special site that is running the latest Gutenberg plugin snapshot and removed [Status] Needs e2e Testing Gutenberg Edge Run e2e tests with a special site that is running the latest Gutenberg plugin snapshot labels Sep 24, 2020
fullofcaffeine and others added 19 commits October 2, 2020 14:33
…laky sleep)"

This reverts commit 7ada50c1613be054d4152123916e65b16a28ac9f.
Doesn't seem to be needed anymore. Saving precious seconds 💥
Gutenberg saves the editor mode in the local storage. When the whole upgrade suite runs in a single browser instnace (i.e when running locally without magellan) it's was bound to fail, since once it re-visited a site, it'd open the editor in the code view, which was unexpected. By deleted the local storage data for the site, once it's re-visited, it then falls-back to the default visual editor, as expected.

Also, extract the code to start a new post into its own helper function since it's used in two places at the moment.
We can go directly to a new post on a target site. No need to login each time as it's a single user for all upgrade test sites.
This reverts commit 54e3c9e1acabb537e4d61d2bcf150dca6be5b2e3.
…editors"

This reverts commit 67a27cd605072e1b6dde2932d2757618da644ab7.
@WunderBart WunderBart force-pushed the add/gb-upgrade-fill-in-blocks-groundwork branch from 3c9441f to 4ad441f Compare October 2, 2020 12:33
@wp-desktop wp-desktop dismissed their stale review October 2, 2020 12:52

wp-desktop ci passing, closing review

@WunderBart WunderBart added the [Status] Needs e2e Testing Gutenberg Edge Run e2e tests with a special site that is running the latest Gutenberg plugin snapshot label Oct 2, 2020
@@ -391,7 +391,7 @@ export default class GutenbergEditorComponent extends AsyncBaseContainer {
* You can just import the class of the block(s) you want to add and pass it to this function, which
* also means we don't need to couple the block class with this one.
*
* @param { typeof GutenbergBlockComponent } blockClass A block class
* @param { Function } blockClass A block class
Copy link
Contributor Author

@fullofcaffeine fullofcaffeine Oct 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one of the types I considered, being that it's the constructor for the class, which is a Function. However, we do miss the actual type narrowing here as this suggests it could be any constructor.

I've been searching around and didn't find a definitive answer for that, but got some hints here: jsdoc/jsdoc#1349, and that's why I chose to use typeof GutenbergBlockComponent.

I'm fine leaving this as Function, though, but it'd be interesting to dive a bit deeper into it later if we have time to figure out if there's a proper syntax for this case.

Copy link
Contributor

@bsessions85 bsessions85 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. thanks for adding!

@fullofcaffeine fullofcaffeine merged commit 712abcc into master Oct 2, 2020
@fullofcaffeine fullofcaffeine deleted the add/gb-upgrade-fill-in-blocks-groundwork branch October 2, 2020 18:25
@matticbot matticbot removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Oct 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Status] Needs e2e Testing Gutenberg Edge Run e2e tests with a special site that is running the latest Gutenberg plugin snapshot
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants