This repository has been archived by the owner on Nov 23, 2023. It is now read-only.
feat: Delete CloudWatch Log Groups over a week old #2338
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Jimlinz
force-pushed
the
cloudwatch-loggroup-cleanup
branch
2 times, most recently
from
a87d342
to
de14a1a
Compare
l0b0
reviewed
Nov 23, 2022
There was a problem hiding this comment.
Code looks good, but I'd want more information in the commit message. Basically all the info in the PR description, ideally including links to relevant issues like aws/aws-cdk#21804.
There is no mechanism in place to clean up CloudWatch Log Groups. While the logs itself can be set to expire, the Log Groups are kept forever. There isn't an elegant way to delete log groups within CDK on stack teardown currently (this may change in the future - see aws/aws-cdk#21804). This is a temporary workaround until a better solution is found. Note: A --no-paginate option is added to limit the number of queries (~50). This prevents rate limiting as well as stops a potentially endless number of results being returned by AWS (which would result in a very long-running workflow). This means only a maximum number of 50 log groups are removed each workflow run.
Jimlinz
force-pushed
the
cloudwatch-loggroup-cleanup
branch
from
de14a1a
to
c131980
Compare
l0b0
approved these changes
Nov 23, 2022
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There is no mechanism in place to clean up CloudWatch Log Groups created by Geostore. While the logs itself can be set to expire, the Log Groups are kept forever. This presents a problem in CI.
There isn't an elegant way to delete log groups within CDK on stack teardown currently (this may change in the future).
This PR runs
aws logs delete-log-groupcommand at the end of the CI to cull log groups older than 1 week old (from previous CI runs). This is a temporary work around until a better solution is found.Note: A
--no-paginateoption is added to limit the number of queries (~50). This prevents rate limiting as well as stops a potentially endless number of results being returned by AWS (which would result in a very log running workflow). This means only a maximum number of 50 log groups are removed each workflow run.A workaround / partial fix to #2319