feat(pacmak): retry select external command invocations #2013
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When generating packages, `jsii-pacmak` will defer to external tools for compilation (`dotnet`, `mvn`, `pip`, ...). Those commands will typically require some kind of network transfer (e.g: to resolve dependencies from their standard package registry), which is susceptible to fail (due to poor network conditions, getting throttled in CI/CD, etc...). In order to reduce the likelihood that such failures will interrupt the user workflow abruptly (forcing a manual retry, and possibly breaking CI/CD workflows), back-off and retries will be performed when these commands fail (up to 5 tentatives in total, with exponential back-off using a 150ms base cooldown and a factor of 2).
RomainMuller
added
effort/small
RomainMuller
force-pushed
the
rmuller/retry-shell-outs
branch
from
88cd919 to
67b4736
Compare
rix0rrr
requested changes
Sep 16, 2020
packages/jsii-pacmak/lib/util.ts
Outdated
| * | ||
| * @default 5 | ||
| */ | ||
| maxTries?: number; |
There was a problem hiding this comment.
maxAttempts then? This word grates on me for some reason.
|
For some reason I only noticed there was already support for retries after writing the new retry wrapper. And the old retry mechanism lacked support for back-off, making it questionnable for working around throttles / DDOS protections. |
rix0rrr
approved these changes
Sep 16, 2020
|
Thank you for contributing! ❤️ I will now look into making sure the PR is up-to-date, then proceed to try and merge it! |
|
Merging (with squash)... |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
|
Merging (with squash)... |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When generating packages,
jsii-pacmakwill defer to external tools forcompilation (
dotnet,mvn,pip, ...). Those commands will typicallyrequire some kind of network transfer (e.g: to resolve dependencies from
their standard package registry), which is susceptible to fail (due to
poor network conditions, getting throttled in CI/CD, etc...).
In order to reduce the likelihood that such failures will interrupt the
user workflow abruptly (forcing a manual retry, and possibly breaking
CI/CD workflows), back-off and retries will be performed when these
commands fail (up to 5 tentatives in total, with exponential back-off
using a 150ms base cooldown and a factor of 2).
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.