Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky ctest tests #6310

Merged
merged 4 commits into from
Jan 28, 2022
Merged

Conversation

sfc-gh-anoyes
Copy link
Collaborator

@sfc-gh-anoyes sfc-gh-anoyes commented Jan 28, 2022

Fix a few sources of flakiness in our "ctest" tests. Flakiness found by running the tests many times in Joshua, using #6237

Code-Reviewer Section

The general guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

  • The PR has a description, explaining both the problem and the solution.
  • The description mentions which forms of testing were done and the testing seems reasonable.
  • Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

  • This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or master if this is the youngest branch)
  • There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

@fdb-windows-ci
Copy link
Collaborator

Doxense CI Report for Windows 10

@foundationdb-ci
Copy link
Contributor

AWS CodeBuild CI Report for macOS BigSur 11.5.2

  • CodeBuild project: foundationdb-pr-macos
  • Commit ID: 134f07a
  • Result: SUCCEEDED
  • Error: N/A
  • Build Logs (available for 30 days)

@foundationdb-ci
Copy link
Contributor

AWS CodeBuild CI Report for Linux CentOS 7

  • CodeBuild project: foundationdb-pr
  • Commit ID: 134f07a
  • Result: SUCCEEDED
  • Error: N/A
  • Build Logs (available for 30 days)

This way we get a predictable 127.0.0.1 in the trace file name
calling fdb_network_stop sends a "close" message to the trace thread,
but the network thread might can still be running and sending "flush"
messages to the network thread. This change basically ignores any
flushes that come after a close.
@@ -216,6 +217,9 @@ def kill(logger):

@enable_logging()
def suspend(logger):
if not shutil.which("pidof"):
logger.debug("Skipping suspend test. Pidof not available")
return
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the case in our CI, such that we will stop running the test there?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember we have it in the CI, not sure if it's changed.
But it's not available on macOS and makes the test fail on macOS.
Add this may make the test work on macOS I guess?

Copy link
Collaborator Author

@sfc-gh-anoyes sfc-gh-anoyes Jan 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the case in joshua agents, but in not CI. We could probably add pidof to the Joshua agents. I'll follow up on that

Copy link
Collaborator

@sfc-gh-clin sfc-gh-clin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing it.

@@ -216,6 +217,9 @@ def kill(logger):

@enable_logging()
def suspend(logger):
if not shutil.which("pidof"):
logger.debug("Skipping suspend test. Pidof not available")
return
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember we have it in the CI, not sure if it's changed.
But it's not available on macOS and makes the test fail on macOS.
Add this may make the test work on macOS I guess?

@foundationdb-ci
Copy link
Contributor

AWS CodeBuild CI Report for macOS BigSur 11.5.2

  • CodeBuild project: foundationdb-pr-macos
  • Commit ID: 25caa34
  • Result: SUCCEEDED
  • Error: N/A
  • Build Logs (available for 30 days)

@fdb-windows-ci
Copy link
Collaborator

Doxense CI Report for Windows 10

@foundationdb-ci
Copy link
Contributor

AWS CodeBuild CI Report for Linux CentOS 7

  • CodeBuild project: foundationdb-pr
  • Commit ID: 25caa34
  • Result: SUCCEEDED
  • Error: N/A
  • Build Logs (available for 30 days)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants