Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metricbeat] Change Account ID to Project ID in gcp.billing module #26412

Merged
merged 5 commits into from
Jun 25, 2021
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -491,6 +491,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Update config in `windows.yml` file. {issue}23027[23027]{pull}23327[23327]
- Fix metric grouping for windows/perfmon module {issue}23489[23489] {pull}23505[23505]
- Major refactor of system/cpu and system/core metrics. {pull}25771[25771]
- Fix GCP Project ID being ingested as `cloud.account.id` in `gcp.billing` module {issue}26357[26357] {pull}26412[26412]

*Packetbeat*

Expand Down
21 changes: 13 additions & 8 deletions x-pack/metricbeat/module/gcp/billing/billing.go
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,8 @@ func (m *MetricSet) queryBigQuery(ctx context.Context, client *bigquery.Client,
SELECT
invoice.month,
project.id,
project.name,
billing_account_id,
cost_type,
(SUM(CAST(cost * 1000000 AS int64))
+ SUM(IFNULL((SELECT SUM(CAST(c.amount * 1000000 as int64)) FROM UNNEST(credits) c), 0))) / 1000000
Expand Down Expand Up @@ -261,26 +263,29 @@ func (m *MetricSet) queryBigQuery(ctx context.Context, client *bigquery.Client,
return events, err
}

if len(row) == 4 {
if len(row) == 6 {
events = append(events, createEvents(row, m.config.ProjectID))
}
}
return events, nil
}

func createEvents(rowItems []bigquery.Value, accountID string) mb.Event {
func createEvents(rowItems []bigquery.Value, projectID string) mb.Event {
event := mb.Event{}
event.MetricSetFields = common.MapStr{
"invoice_month": rowItems[0],
"project_id": rowItems[1],
"cost_type": rowItems[2],
"total": rowItems[3],
"invoice_month": rowItems[0],
"project_id": rowItems[1],
"project_name": rowItems[3],
legoguy1000 marked this conversation as resolved.
Show resolved Hide resolved
"billing_account_id": rowItems[4],
"cost_type": rowItems[5],
"total": rowItems[9],
}

event.RootFields = common.MapStr{
"cloud.provider": "gcp",
"cloud.account.id": accountID,
"cloud.account.name": accountID,
"cloud.project.id": projectID,
"cloud.project.name": rowItems[3],
"cloud.account.id": rowItems[4],
}

// create eventID for each current_date + invoice_month + project_id + cost_type
Expand Down