Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTML API: Improve skipped test reporting with unsupported exception #7285

Closed

Conversation

sirreal
Copy link
Member

@sirreal sirreal commented Sep 3, 2024

Trac ticket: Core-61646

Improve unsupported markup skipped test information.

The html5lib-tests suite skips a number of tests due to unsupported markup. At the moment, these tests all report "Test includes unsupported markup." Use the get_unsupported_exception method to improve these skip messages so they're more informative: "Unsupported markup: Foster parenting is not supported."

There is no diff in the test statistics comparing this branch to trunk.

To test this, run the following and the improved error messages will be shown:

phpunit --group html-api-html5lib-tests --verbose

Here are a selection of test outputs for skipped tests.

Before
Screenshot 2024-09-03 at 11 16 25 AM

After
Screenshot 2024-09-03 at 11 15 55 AM


This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

@sirreal
Copy link
Member Author

sirreal commented Sep 3, 2024

Example

36) Tests_HtmlApi_Html5lib::test_parse with data set "tests1/line1149" (null, '<a href="blah">aba<table><a h...le>aoe', '<html>\n  <head>\n  <body>\n ...e"\n\n')
Test includes unsupported markup.

Copy link

github-actions bot commented Sep 3, 2024

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

  • The Plugin and Theme Directories cannot be accessed within Playground.
  • All changes will be lost when closing a tab with a Playground instance.
  • All changes will be lost when refreshing the page.
  • A fresh instance is created each time the link below is clicked.
  • Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
    it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

@sirreal sirreal marked this pull request as ready for review September 3, 2024 14:13
Copy link

github-actions bot commented Sep 3, 2024

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props jonsurrell, dmsnell.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

} catch ( WP_HTML_Unsupported_Exception $e ) {
$this->markTestSkipped( "Unsupported markup: {$e->getMessage()}" );
return;
}

if ( null === $processed_tree ) {
$this->markTestSkipped( 'Test includes unsupported markup.' );
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably inaccurate now, the unsupported markup messages should appear above.

I'd like to improve the other error cases because returning null can't express all of them. I've been thinking about adding more classes to handle them.

@dmsnell
Copy link
Member

dmsnell commented Sep 3, 2024

@sirreal I added new sections to throw an error on incomplete input and on an error that isn't unsupported markup (e.g. out of bookmarks)

Copy link
Member

@dmsnell dmsnell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sirreal this is great - gives us better information on unsupported failures.

pento pushed a commit that referenced this pull request Sep 3, 2024
The `html5lib-tests` suite skips a number of tests due to unsupported markup. At the moment, these tests all report "Test includes unsupported markup." This patch calls the `get_unsupported_exception()` method in these skipped cases to improve the messages reported to PHPUnit so they're more informative: e.g. "Unsupported markup: Foster parenting is not supported."

Developed in #7285
Discussed in https://core.trac.wordpress.org/ticket/61646

Follow-up to [58714].

Props dmsnell, jonsurrell.
See #61646.


git-svn-id: https://develop.svn.wordpress.org/trunk@58971 602fd350-edb4-49c9-b593-d223f7449a82
@dmsnell
Copy link
Member

dmsnell commented Sep 3, 2024

Merged in [58971]
f79de00

@dmsnell dmsnell closed this Sep 3, 2024
markjaquith pushed a commit to WordPress/WordPress that referenced this pull request Sep 3, 2024
The `html5lib-tests` suite skips a number of tests due to unsupported markup. At the moment, these tests all report "Test includes unsupported markup." This patch calls the `get_unsupported_exception()` method in these skipped cases to improve the messages reported to PHPUnit so they're more informative: e.g. "Unsupported markup: Foster parenting is not supported."

Developed in WordPress/wordpress-develop#7285
Discussed in https://core.trac.wordpress.org/ticket/61646

Follow-up to [58714].

Props dmsnell, jonsurrell.
See #61646.

Built from https://develop.svn.wordpress.org/trunk@58971


git-svn-id: http://core.svn.wordpress.org/trunk@58367 1a063a9b-81f0-0310-95a4-ce76da25c4cd
github-actions bot pushed a commit to gilzow/wordpress-performance that referenced this pull request Sep 3, 2024
The `html5lib-tests` suite skips a number of tests due to unsupported markup. At the moment, these tests all report "Test includes unsupported markup." This patch calls the `get_unsupported_exception()` method in these skipped cases to improve the messages reported to PHPUnit so they're more informative: e.g. "Unsupported markup: Foster parenting is not supported."

Developed in WordPress/wordpress-develop#7285
Discussed in https://core.trac.wordpress.org/ticket/61646

Follow-up to [58714].

Props dmsnell, jonsurrell.
See #61646.

Built from https://develop.svn.wordpress.org/trunk@58971


git-svn-id: https://core.svn.wordpress.org/trunk@58367 1a063a9b-81f0-0310-95a4-ce76da25c4cd
@sirreal sirreal deleted the html-api/try-better-skip-reporting branch September 5, 2024 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants