Add generic end 2 end test to all block transforms #10536

jorgefilipecosta · 2018-10-11T21:21:16Z

Description

This PR proposes a generic approach to test the block transforms we have.
Fist it loads all the fixtures of content we have and checks if the transforms that should appear on the block they represent correctly appear.
Then it checks in a loop if each transform contains the output that should is expected.
If in the future a new block is created a new fixture should create for it. If that block contains transforms, we should update our fixtures/block-transforms.json specifying which transforms are available and a new test case is automatically generated and a snapshot created. if the fixtures/block-transforms.json file is not updated the tests will fail so this makes sure the tests do not get outdated, each transform is tested.

ellatrix · 2018-10-12T11:19:19Z

test/integration/full-content/fixtures/core__gallery.html

@@ -7,7 +7,7 @@
 	</li>
 	<li class="blocks-gallery-item">
 		<figure>
-			<img src="http://google.com/hi.png" alt="title" />
+			<img src="https://cldup.com/cXyG__fTLN.jpg" alt="title" />


Could you explain this change? Thanks!

Oh I see now, never mind. Would be cool if the image moved so it's in our control.

Yes, that's true but what reliable source can we consider under our control?
I used cldup here as we are also using it on the demo page so at least we only depend on this domain.

ellatrix · 2018-10-12T11:20:47Z

test/integration/full-content/fixtures/core__video.html

@@ -1,3 +1,3 @@
 <!-- wp:core/video -->
-<figure class="wp-block-video"><video controls src="https://awesome-fake.video/file.mp4"></video></figure>
+<figure class="wp-block-video"><video controls src="https://www.sample-videos.com/video/mp4/720/big_buck_bunny_720p_1mb.mp4"></video></figure>


Same here. We have no control over sample-videos.com? What if something happens with that site?

I will switch to a cldup url.

I don't know about cldup either tbh. I think any files should be in this repo. Test images an files could be super tiny.

I guess it can be cloud up for now as we're already doing that.

Tiny files might speed up the tests as well though.

Idea: https://github.com/mathiasbynens/small.

I like this idea, but I'm not certain how we can make sure that a local URL for a file is available. The other option would use a data URL that contains the files inline but for example, a video still contains many characters.

Apparently it would be:

data:video/mp4;base64,AAAAHGZ0eXBpc29tAAACAGlzb21pc28ybXA0MQAAAAhmcmVlAAAAGm1kYXQAAAGzABAHAAABthBgUYI9t+8AAAMNbW9vdgAAAGxtdmhkAAAAAMXMvvrFzL76AAAD6AAAACoAAQAAAQAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAABhpb2RzAAAAABCAgIAHAE/////+/wAAAiF0cmFrAAAAXHRraGQAAAAPxcy++sXMvvoAAAABAAAAAAAAACoAAAAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAAgAAAAIAAAAAAG9bWRpYQAAACBtZGhkAAAAAMXMvvrFzL76AAAAGAAAAAEVxwAAAAAALWhkbHIAAAAAAAAAAHZpZGUAAAAAAAAAAAAAAABWaWRlb0hhbmRsZXIAAAABaG1pbmYAAAAUdm1oZAAAAAEAAAAAAAAAAAAAACRkaW5mAAAAHGRyZWYAAAAAAAAAAQAAAAx1cmwgAAAAAQAAAShzdGJsAAAAxHN0c2QAAAAAAAAAAQAAALRtcDR2AAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAgACABIAAAASAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGP//AAAAXmVzZHMAAAAAA4CAgE0AAQAEgICAPyARAAAAAAMNQAAAAAAFgICALQAAAbABAAABtYkTAAABAAAAASAAxI2IAMUARAEUQwAAAbJMYXZjNTMuMzUuMAaAgIABAgAAABhzdHRzAAAAAAAAAAEAAAABAAAAAQAAABxzdHNjAAAAAAAAAAEAAAABAAAAAQAAAAEAAAAUc3RzegAAAAAAAAASAAAAAQAAABRzdGNvAAAAAAAAAAEAAAAsAAAAYHVkdGEAAABYbWV0YQAAAAAAAAAhaGRscgAAAAAAAAAAbWRpcmFwcGwAAAAAAAAAAAAAAAAraWxzdAAAACOpdG9vAAAAG2RhdGEAAAABAAAAAExhdmY1My4yMS4x

Not a small amount of characters, but it still might have a big impact on test speed and safeness.

Image: data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7

ellatrix · 2018-10-12T11:22:11Z

test/e2e/specs/block-transforms.test.js

+};
+
+describe( 'test transforms', () => {
+	const expectedTransforms = JSON.parse(


Could we put this in a .js file?

ellatrix · 2018-10-12T11:27:58Z

test/e2e/specs/__snapshots__/block-transforms.test.js.snap

@@ -0,0 +1,297 @@
+// Jest Snapshot v1, https://goo.gl/fbAQLP
+
+exports[`test transforms individual transforms work as expected Should correctly transform block in fixture core__audio to File block 1`] = `


Is there any way to make this more descriptive and concise? Something like: "Paragraph block should transform to Heading block"?

We can do that, but the thing is that we have many tests testing Paragraph to Heading block (one for each paragraph fixture), that was the reason I added the fixture to make it easier to debug in case the test fails. Maybe adding also the block name is a good trade-off.

Ok, we can always iterate on this.

ellatrix · 2018-10-12T11:35:29Z

test/e2e/specs/block-transforms.test.js

+				fileBase
+			);
+		}
+		expect( console ).toHaveErroredWith(


Why are we expecting this? Could this be commented?

ellatrix · 2018-10-12T11:36:45Z

test/e2e/specs/block-transforms.test.js

+	const transformStructure = {};
+	beforeAll( async () => {
+		await newPost();
+		await page.click( '.editor-post-title .editor-post-title__block' );


Why is this needed? Just wondering.

ellatrix · 2018-10-12T11:37:38Z

test/e2e/specs/block-transforms.test.js

+		);
+	} );
+
+	it( 'Should contain the expected transforms', async () => {


Generally I think we should not capitalise "should" because it will be placed in a sentence.

ellatrix · 2018-10-12T11:38:31Z

test/e2e/specs/block-transforms.test.js

+	describe( 'individual transforms work as expected', () => {
+		afterEach( async () => {
+			await setPostContent( '' );
+			await page.click( '.editor-post-title .editor-post-title__block' );


After reading beforeAll, it should like this should happen beforeEach?

ellatrix · 2018-10-12T11:39:57Z

test/e2e/specs/fixtures/block-transforms.json

+	"core__archives":[],
+	"core__archives__showPostCounts":[],
+	"core__audio":[
+		"File"


Any way to have this both be block names (plus attributes) or both be file names?

ellatrix · 2018-10-12T11:41:17Z

test/support/utils.js

@@ -0,0 +1,32 @@
+/**


Is there any of this that could be shared with full-content.spec.js?

jorgefilipecosta · 2018-10-12T20:34:27Z

Hi @iseulde, thank you a lot for your review, I think all the feedback was applied.

Squashed commits: [fd5c73823] Ignore missing block. [0d9ef5e70] Updated fixtures. [9d04f9909] Used utils function in full-content [321564b94] Improvements [fe4bd6fbc] Console errors fixed seems to be working without expecting any error. [a4f802ca1] Finalized implementation. [79a135b3d] Add end 2 end test block transforms.

jorgefilipecosta · 2018-11-26T22:45:55Z

Closed in favor of #12336.

@gziolo

## Description This is a second try on PR #10536, after a recommendation by @gziolo to try to make the PR smaller so it becomes easier to review. In order to make the PR smaller, I temporarily created a list of test fixtures that intersects with the set of all fixtures so most fixtures are ignored. Another PR will follow that corrects the fixtures (mainly media ones) so they are reliable and can be used in this tests (e.g: don't trigger 404 requests). All the mechanisms are present in this PR the only change that will happen in the other PR is the removal of the intersection and its list, the update of some media fixtures and the creation of new snapshots. This is a high-level test that is going to Guarantee that every possible transform on Gutenberg is tested and works as expected (the result matches the save snapshot). ## How has this been tested? Verify all test cases pass.

jorgefilipecosta added the [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. label Oct 11, 2018

jorgefilipecosta force-pushed the add/end-2-end-tests-block-transforms branch 2 times, most recently from d76d1b1 to a013ad4 Compare October 11, 2018 22:07

jorgefilipecosta mentioned this pull request Oct 12, 2018

Add list and quote block transformation e2e tests #10507

Merged

4 tasks

ellatrix reviewed Oct 12, 2018

View reviewed changes

jorgefilipecosta force-pushed the add/end-2-end-tests-block-transforms branch 2 times, most recently from b5aaec7 to 7ffc1dc Compare October 12, 2018 20:16

jorgefilipecosta force-pushed the add/end-2-end-tests-block-transforms branch 2 times, most recently from 2d6c12a to 822a947 Compare November 1, 2018 13:14

jorgefilipecosta force-pushed the add/end-2-end-tests-block-transforms branch from 822a947 to d0718f2 Compare November 1, 2018 16:10

jorgefilipecosta requested a review from a team November 9, 2018 00:17

jorgefilipecosta mentioned this pull request Nov 26, 2018

Add: Generic end 2 end test to the block transforms. #12336

Merged

jorgefilipecosta closed this Nov 26, 2018

youknowriad deleted the add/end-2-end-tests-block-transforms branch November 27, 2018 07:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add generic end 2 end test to all block transforms #10536

Add generic end 2 end test to all block transforms #10536

jorgefilipecosta commented Oct 11, 2018 •

edited

Loading

ellatrix Oct 12, 2018

ellatrix Oct 12, 2018

jorgefilipecosta Oct 12, 2018

ellatrix Oct 12, 2018

jorgefilipecosta Oct 12, 2018

ellatrix Oct 12, 2018

ellatrix Oct 12, 2018

ellatrix Oct 12, 2018

ellatrix Oct 12, 2018

jorgefilipecosta Oct 12, 2018

ellatrix Oct 12, 2018

ellatrix Oct 12, 2018 •

edited

Loading

ellatrix Oct 12, 2018

ellatrix Oct 12, 2018

ellatrix Oct 12, 2018

jorgefilipecosta Oct 12, 2018

ellatrix Oct 12, 2018

ellatrix Oct 12, 2018

ellatrix Oct 12, 2018

ellatrix Oct 12, 2018

ellatrix Oct 12, 2018

ellatrix Oct 12, 2018

ellatrix Oct 12, 2018

jorgefilipecosta commented Oct 12, 2018

jorgefilipecosta commented Nov 26, 2018

		@@ -0,0 +1,297 @@
		// Jest Snapshot v1, https://goo.gl/fbAQLP

		exports[`test transforms individual transforms work as expected Should correctly transform block in fixture core__audio to File block 1`] = `

Add generic end 2 end test to all block transforms #10536

Add generic end 2 end test to all block transforms #10536

Conversation

jorgefilipecosta commented Oct 11, 2018 • edited Loading

Description

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ellatrix Oct 12, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorgefilipecosta commented Oct 12, 2018

jorgefilipecosta commented Nov 26, 2018

jorgefilipecosta commented Oct 11, 2018 •

edited

Loading

ellatrix Oct 12, 2018 •

edited

Loading