Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Alert for Errors in API Application Logs #1475

Merged
merged 7 commits into from
Mar 26, 2024
Merged

Conversation

aplybeah
Copy link
Contributor

@aplybeah aplybeah commented Mar 13, 2024

Summary

Fixes #1033

Time to review: x mins

Changes proposed

  • add metric filter
  • add alarm based on that filter

Context for reviewers

There are not currently alerts for errors in the API applications logs. The application is currently logged in service/api-prod

Opted to change strategies last minute because the metric filter is slightly better at tracing things in unstructured errors.

Additional information

Metric filter from the cloudwatch log group side:
Screen Shot 2024-03-26 at 2 09 56 PM
Screen Shot 2024-03-26 at 2 09 38 PM

the alarm:
Screen Shot 2024-03-26 at 2 14 24 PM

@aplybeah aplybeah marked this pull request as ready for review March 26, 2024 18:15
@aplybeah aplybeah requested a review from acouch March 26, 2024 18:15
Copy link
Collaborator

@coilysiren coilysiren left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New fancy alarms!

resource "aws_cloudwatch_log_metric_filter" "service_error_filter" {

name = "service-error-filter"
pattern = "%ERROR%" # pattern can find events in unstructured logs
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⭐⭐   (optional but seriously consider)

Nice! It this a regex? Can we update it to (ERROR|Exception)?

I feel like exception is more commonly going to be in the message than error is. In python, at least

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is! I found this which was super helpful

@aplybeah aplybeah merged commit 8c21a7b into main Mar 26, 2024
7 checks passed
@aplybeah aplybeah deleted the alsia/app-error-alarms branch March 26, 2024 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Task]: Create Alert for Errors in API Application Logs
2 participants