Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Merged by Bors] - Correctly run async tests #2683

Closed
wants to merge 5 commits into from
Closed

[Merged by Bors] - Correctly run async tests #2683

wants to merge 5 commits into from

Conversation

jedel1043
Copy link
Member

So, while working on #2658 I apparently broke 1280 tests... except I didn't! Turns out we were considering async tests that didn't call the print function as passed tests, but the async section of https://github.com/tc39/test262/blob/main/INTERPRETING.md#flags says:

The test must not be considered complete until the implementation-defined print function has been invoked or some length of time has passed without any such invocation.

@jedel1043 jedel1043 added bug Something isn't working test Issues and PRs related to the tests. labels Mar 16, 2023
@jedel1043 jedel1043 added this to the v0.17.0 milestone Mar 16, 2023
@codecov
Copy link

codecov bot commented Mar 16, 2023 •
edited
Loading

Codecov Report

Merging #2683 (99800de) into main (c9ae4dc) will increase coverage by 0.31%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main    #2683      +/-   ##
==========================================
+ Coverage   49.17%   49.49%   +0.31%     
==========================================
  Files         397      393       -4     
  Lines       39748    39473     -275     
==========================================
- Hits        19545    19536       -9     
+ Misses      20203    19937     -266
Impacted Files Coverage Δ
boa_tester/src/exec/mod.rs 0.00% <0.00%> (ø)

... and 12 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@jedel1043
Copy link
Member Author

Test262 conformance changes

Test result main count PR count difference
Total 94,277 94,277 0
Passed 71,994 70,714 -1,280
Ignored 17,324 17,324 0
Failed 4,959 6,239 +1,280
Panics 14 14 0
Conformance 76.36% 75.01% -1.36%
Broken tests (1280): 
(1280 tests)

nekevss
nekevss reviewed Mar 17, 2023
View reviewed changes
boa_tester/src/exec/mod.rs Outdated Show resolved Hide resolved
Razican
Razican approved these changes Mar 17, 2023
View reviewed changes
Copy link
Member

@Razican Razican left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! I think we can merge it with a bit of documentation in the new result structure.

boa_tester/src/exec/mod.rs Show resolved Hide resolved
@jedel1043 jedel1043 requested a review from nekevss March 17, 2023 06:35
@jedel1043 jedel1043 mentioned this pull request Mar 17, 2023
Closed
HalidOdat
HalidOdat approved these changes Mar 17, 2023
View reviewed changes
Copy link
Member

@HalidOdat HalidOdat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! :)

@Razican
Copy link
Member

Razican commented Mar 17, 2023

bors r+

bors bot pushed a commit that referenced this pull request Mar 17, 2023
@jedel1043
Correctly run async tests (#2683)
a53edeb 
So, while working on #2658 I apparently broke 1280 tests... except I didn't! Turns out we were considering async tests that didn't call the `print` function as passed tests, but the `async` section of https://github.com/tc39/test262/blob/main/INTERPRETING.md#flags says:
> The test must not be considered complete until the implementation-defined print function has been invoked or some length of time has passed without any such invocation.
@bors
Copy link

bors bot commented Mar 17, 2023

Pull request successfully merged into main.

Build succeeded:

@bors bors bot changed the title Correctly run async tests [Merged by Bors] - Correctly run async tests Mar 17, 2023
@bors bors bot closed this Mar 17, 2023
@bors bors bot deleted the good-async-tests branch March 17, 2023 16:44
bors bot pushed a commit that referenced this pull request Mar 21, 2023
@jedel1043
Align iterator loops to the spec (#2686)
cb4e49a 
~Depends on #2683.~ Merged.

This Pull Request fixes #2658.

It changes the following:

- Makes `for .. of` loop execution more spec compliant.
- Rewrites iterator related opcodes to be able to use it on all for .. of/in loops.
- Adds some utility op codes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HalidOdat HalidOdat HalidOdat approved these changes

@Razican Razican Razican approved these changes

@jasonwilliams jasonwilliams Awaiting requested review from jasonwilliams

@RageKnify RageKnify Awaiting requested review from RageKnify

@raskad raskad Awaiting requested review from raskad

@nekevss nekevss Awaiting requested review from nekevss

Assignees
No one assigned
Labels
bug Something isn't working test Issues and PRs related to the tests.
Projects
None yet
Milestone
v0.17.0
Development

Successfully merging this pull request may close these issues.
4 participants
@jedel1043 @Razican @HalidOdat @nekevss