Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_runner: fix todo and only in spec reporter #48929

Closed
wants to merge 2 commits into from
Closed

test_runner: fix todo and only in spec reporter #48929

wants to merge 2 commits into from

Conversation

MoLow
Copy link
Member

@MoLow MoLow commented Jul 26, 2023

No description provided.

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Jul 26, 2023
@MoLow MoLow force-pushed the test-runner-fix-todo-and-only branch from fe46cc8 to 900724d Compare July 26, 2023 11:43
cjihrig
cjihrig reviewed Jul 26, 2023
View reviewed changes
lib/internal/test_runner/test.js Outdated
@@ -259,7 +259,7 @@ class Test extends AsyncResource {
skip = '\'only\' option not set';
}

if (skip) {
if (skip || todo) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you making a TODO into a noop? A TODO test should still run, but not fail the test suite if the test fails.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, I think it never has behaved that way :/ anyway I,l revert this part and keep only the reporter fixes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just verified that todo tests show up as todo in the summary, but incorrectly set the process exit code.

Copy link
Member Author

@MoLow MoLow Jul 26, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've pushed a fix in a separate commit

@MoLow MoLow force-pushed the test-runner-fix-todo-and-only branch from 900724d to e1e795c Compare July 26, 2023 19:19
@MoLow MoLow requested a review from cjihrig July 26, 2023 19:20
@MoLow MoLow added the commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. label Jul 26, 2023
cjihrig
cjihrig approved these changes Jul 26, 2023
View reviewed changes
Copy link
Contributor

@cjihrig cjihrig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

By the way, if anyone reading this is interested in doing more TODO test work, the TAP spec mentions that these tests are intended to fail and if they start passing, the test runner may report that in some way so that the user can drop the TODO designation. I think we already have all of the machinery in place to implement that fairly easily.

lib/internal/main/test_runner.js Show resolved Hide resolved
test/fixtures/test-runner/output/describe_it.js Outdated Show resolved Hide resolved
test/fixtures/test-runner/output/describe_it.js Outdated Show resolved Hide resolved
@MoLow MoLow force-pushed the test-runner-fix-todo-and-only branch from e1e795c to e8c3f01 Compare July 26, 2023 19:58
@MoLow MoLow requested a review from cjihrig July 26, 2023 19:58
@MoLow MoLow added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. request-ci Add this label to start a Jenkins CI on a PR. labels Jul 26, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 26, 2023
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/52945/

@github-actions github-actions bot mentioned this pull request Jul 27, 2023
Open
18 tasks
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/52950/

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/52952/

This was referenced Jul 28, 2023
Open
Open
This was referenced Jul 30, 2023
Open
Open
Open
Open
pluris pushed a commit to pluris/node that referenced this pull request Aug 6, 2023
@MoLow
test_runner: fix todo and only in spec reporter
a3375af 
PR-URL: nodejs#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
pluris pushed a commit to pluris/node that referenced this pull request Aug 6, 2023
@MoLow
test_runner: dont set exit code on todo tests
eb7d7b7 
PR-URL: nodejs#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
pluris pushed a commit to pluris/node that referenced this pull request Aug 7, 2023
@MoLow
test_runner: fix todo and only in spec reporter
625fb47 
PR-URL: nodejs#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
pluris pushed a commit to pluris/node that referenced this pull request Aug 7, 2023
@MoLow
test_runner: dont set exit code on todo tests
5f5924b 
PR-URL: nodejs#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
@MoLow MoLow mentioned this pull request Aug 10, 2023
Closed
Ceres6 pushed a commit to Ceres6/node that referenced this pull request Aug 14, 2023
@MoLow @Ceres6
test_runner: fix todo and only in spec reporter
fde0b39 
PR-URL: nodejs#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Ceres6 pushed a commit to Ceres6/node that referenced this pull request Aug 14, 2023
@MoLow @Ceres6
test_runner: dont set exit code on todo tests
21befee 
PR-URL: nodejs#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Ceres6 pushed a commit to Ceres6/node that referenced this pull request Aug 14, 2023
@MoLow @Ceres6
test_runner: fix todo and only in spec reporter
95e271e 
PR-URL: nodejs#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Ceres6 pushed a commit to Ceres6/node that referenced this pull request Aug 14, 2023
@MoLow @Ceres6
test_runner: dont set exit code on todo tests
74622bf 
PR-URL: nodejs#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
UlisesGascon pushed a commit to UlisesGascon/node that referenced this pull request Aug 14, 2023
@MoLow @UlisesGascon
test_runner: fix todo and only in spec reporter
7950f59 
PR-URL: nodejs#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
UlisesGascon pushed a commit to UlisesGascon/node that referenced this pull request Aug 14, 2023
@MoLow @UlisesGascon
test_runner: dont set exit code on todo tests
d919d76 
PR-URL: nodejs#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
RafaelGSS pushed a commit that referenced this pull request Aug 15, 2023
@MoLow @RafaelGSS
test_runner: fix todo and only in spec reporter
0324c80 
PR-URL: #48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
RafaelGSS pushed a commit that referenced this pull request Aug 15, 2023
@MoLow @RafaelGSS
test_runner: dont set exit code on todo tests
56d0958 
PR-URL: #48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
@UlisesGascon UlisesGascon mentioned this pull request Aug 15, 2023
Merged
RafaelGSS pushed a commit to RafaelGSS/node that referenced this pull request Aug 15, 2023
@MoLow @RafaelGSS
test_runner: fix todo and only in spec reporter
20269a0 
PR-URL: nodejs#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
RafaelGSS pushed a commit to RafaelGSS/node that referenced this pull request Aug 15, 2023
@MoLow @RafaelGSS
test_runner: dont set exit code on todo tests
454d8dc 
PR-URL: nodejs#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
RafaelGSS pushed a commit that referenced this pull request Aug 17, 2023
@MoLow @RafaelGSS
test_runner: fix todo and only in spec reporter
52c9490 
PR-URL: #48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
RafaelGSS pushed a commit that referenced this pull request Aug 17, 2023
@MoLow @RafaelGSS
test_runner: dont set exit code on todo tests
9136667 
PR-URL: #48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
@cjihrig cjihrig mentioned this pull request Oct 23, 2023
Closed
targos pushed a commit that referenced this pull request Nov 27, 2023
@MoLow @targos
test_runner: fix todo and only in spec reporter
d217435 
PR-URL: #48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
targos pushed a commit that referenced this pull request Nov 27, 2023
@MoLow @targos
test_runner: dont set exit code on todo tests
00c70e1 
PR-URL: #48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
@targos targos mentioned this pull request Nov 28, 2023
Merged
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
test_runner: fix todo and only in spec reporter
44483cc 
PR-URL: nodejs/node#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
test_runner: dont set exit code on todo tests
e8d4aab 
PR-URL: nodejs/node#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
test_runner: fix todo and only in spec reporter
aacf0c0 
PR-URL: nodejs/node#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
test_runner: dont set exit code on todo tests
00d32a2 
PR-URL: nodejs/node#48929
Reviewed-By: Chemi Atlow <chemi@atlow.co.il>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@atlowChemi atlowChemi atlowChemi approved these changes

@cjihrig cjihrig cjihrig approved these changes

@benjamingr benjamingr benjamingr approved these changes

Assignees
No one assigned
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@MoLow @nodejs-github-bot @benjamingr @cjihrig @atlowChemi