fix(testing): fix race condition in tests for parallel commands runner #2358
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We are now checking for the presence of both command outputs regardless of the order they were written in to the file
… parallel In addition to checking whether the tasks complete and they write to the file, we are now also checking if the runner creates all the tasks synchronously when invoked with parallel: true, vs. zero or at most 1 when ran in serial mode.
FrozenPandaz
approved these changes
Jan 24, 2020
| } | ||
| ], | ||
| parallel: true | ||
| } | ||
| ); | ||
| const processesCreated = exec.calls.count(); |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
There was a problem hiding this comment.
Is this comment in the right place? :)
There was a problem hiding this comment.
I added your suggestion to the serial runner tests 👍 please let me know if that's not what you meant
Due to differences in implementation between serial and parallel runner, we need to wait for a micro-tick in the serial test runner tests before checking for calls to spawn-process, otherwise it would be zero.
FrozenPandaz
approved these changes
Jan 24, 2020
|
This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Current Behavior (This is the behavior we have today, before the PR is merged)
Test would occasionally fail.
Expected Behavior (This is the new behavior we can expect after the PR is merged)
Test should consistently pass now.
Issue
The previous setup was as follows:
The idea was that, if truly running in parallel, the second commands would finish before the first, because it was sleeping for less.
But as @jaysoo pointed out, if they're running in parallel we can't guarantee order, even with
sleep.Fix (checked-in / part of this PR)
Instead of checking the order in which the commands were executed we now:
To prove above strategy works, it's also been added to the
serialrunner tests, and it indeed just prints 0 (as the initial promise it creates is a null one, only once that resolves it schedules the first task in the queue)