Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Metrics: TotalRecords Improvement #527

Closed
jacknagz opened this issue Dec 13, 2017 · 0 comments · Fixed by #548
Closed

Metrics: TotalRecords Improvement #527

jacknagz opened this issue Dec 13, 2017 · 0 comments · Fixed by #548
Labels
bug lambda metrics

Comments

@jacknagz
Copy link
Contributor

Background

Description

Currently, the TotalRecords metric is based around the number of records in the event which invoked Lambda:

streamalert/stream_alert/rule_processor/handler.py

Line 119 in 560c0f7

MetricLogger.log_metric(FUNCTION_NAME, MetricLogger.TOTAL_RECORDS, len(records))

This is slightly misleading when records are nested, such as with VPC flow logs, or perhaps with S3 files that are pulled and parsed.

Steps to Reproduce

Observe your Lambda function execution for nested records, and this pattern will show as metrics are logged to CloudWatch.

Desired Change

Base the TotalRecords metric on the number of records parsed into the StreamPayload object:

streamalert/stream_alert/rule_processor/handler.py

Line 381 in 560c0f7

len(payload.records),

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug lambda metrics
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[metrics] getting a more accurate processed record count in rp handler airbnb/streamalert
1 participant
@jacknagz