Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate flaky test eval_messages #2672

Closed
orangemocha opened this issue Sep 3, 2015 · 1 comment
Closed

Investigate flaky test eval_messages #2672

orangemocha opened this issue Sep 3, 2015 · 1 comment
Labels
arm Issues and PRs related to the ARM platform. test Issues and PRs related to the tests.
Milestone

Comments

@orangemocha
Copy link
Contributor

As seen in #2620 (comment)

https://ci.nodejs.org/job/node-test-commit-arm/434/nodes=armv7-wheezy/tapTestReport/

not ok 838 - eval_messages.js
# [eval]
# [eval]:1
# with(this){__filename}
# ^^^^
# 
# SyntaxError: Strict mode code may not include a with statement
# at Object.exports.runInThisContext (vm.js:53:16)
# at Object.<anonymous> ([eval]-wrapper:6:22)
# at Module._compile (module.js:430:26)
# at node.js:566:27
# at doNTCallback0 (node.js:407:9)
# at process._tickCallback (node.js:336:13)
#42
#42
@orangemocha orangemocha added the test Issues and PRs related to the tests. label Sep 3, 2015
@brendanashworth brendanashworth added smoketest and removed test Issues and PRs related to the tests. labels Sep 3, 2015
@orangemocha orangemocha added this to the 5.0.0 milestone Sep 3, 2015
@brendanashworth brendanashworth added test Issues and PRs related to the tests. arm Issues and PRs related to the ARM platform. and removed smoketest labels Sep 3, 2015
@rvagg rvagg mentioned this issue Oct 16, 2015
13 tasks
@Trott
Copy link
Member

Trott commented Oct 18, 2015

In the six and a half weeks since the build that this test failed on armv7-wheezy, it has not failed again on armv7-wheezy. And it had apparently not failed before that either. Any objections to removing its flaky status? It seems to me that either one of two things are happening:

  • This was caused by some weird momentary external thing, perhaps with the CI infrastructure, that we will likely never be able to reproduce.
  • Technically, the test is flaky, but it's flaky with such unbelievable infrequency that it is in fact indistinguishable from not flaky.

EDIT: There's also the possibility that it was a problem with the build (e.g., a configuration or OS bug) that has since been corrected. But whatever the case, it seems that there's perhaps not much point in having it marked flaky. If the test fails again, it seems that it is more likely the result of an actual bug in the commit and I'd rather have it flagged than marked as "oh yeah, that test failed, but it fails a lot so you can ignore it".

Trott added a commit to Trott/io.js that referenced this issue Oct 18, 2015
This test has not failed on armv7-wheezy in over 6 weeks. Let's remove its
"flaky" status.

Fixes: nodejs#2672
@orangemocha orangemocha modified the milestones: 6.0.0, 5.0.0 Oct 19, 2015
@Trott Trott closed this as completed in 7cb89f5 Oct 20, 2015
Trott added a commit that referenced this issue Oct 21, 2015
This test has not failed on armv7-wheezy in over 6 weeks. Let's remove its
"flaky" status.

Fixes: #2672
Reviewed-By: Rod Vagg <r@va.gg>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: #3420
Trott added a commit that referenced this issue Oct 26, 2015
This test has not failed on armv7-wheezy in over 6 weeks. Let's remove its
"flaky" status.

Fixes: #2672
Reviewed-By: Rod Vagg <r@va.gg>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: #3420
Trott added a commit that referenced this issue Oct 29, 2015
This test has not failed on armv7-wheezy in over 6 weeks. Let's remove its
"flaky" status.

Fixes: #2672
Reviewed-By: Rod Vagg <r@va.gg>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: #3420
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arm Issues and PRs related to the ARM platform. test Issues and PRs related to the tests.
Projects
None yet
Development

No branches or pull requests

3 participants