Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multiple values per one cost filter in aws_budgets_budget resource #9092

Merged
merged 15 commits into from
Jul 27, 2021

Conversation

mozamimy
Copy link
Contributor

@mozamimy mozamimy commented Jun 22, 2019

Current aws_budgets_budget resource does not support multiple cost filter values because the cost_filters map takes only one string value per string key. However, AWS Budgets API and its console can handle multiple values per one cost filter key.

This change makes it to be able to handle multiple cost filter values like following syntax. This is a similar concept to parameter attribute of Resource: aws_db_parameter_group.

resource "aws_budgets_budget" "monthly_cost_budget" {
  # :
  # (snip)
  # :

  cost_filter {
    name = "Region"
    values = [
      "ap-northeast-1",
      "us-east-1",
    ]
  }

  cost_filter {
    name = "PurchaseType"
    values = [
      "Spot",
    ]
  }

  # :
  # (snip)
  # :
}

I tried to implement like "Potential Terraform Configuration" section of #5890 but I couldn't. I suppose the value of TypeMap should be primitive type (it means that we cannot use TypeList as the TypeMap's value) by the reason reported https://github.com/hashicorp/terraform/issues/15327 .

Of course, the cost_filter attributes conflicts with cost_filters. Thus I think its good to deprecate cost_filters attribute.

Would you review my change and if my idea is correct?

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Closes #5890.
Closes #10692.
Closes #12669.
Closes #13288.
Closes #19803.

Release note for CHANGELOG:

Support multiple cost filter values by adding the `cost_filter` attribute and depreciating the `cost_filters` attribute

Output from acceptance testing:

[14:32:08]mozamimy@ip-10-33-164-61:terraform-provider-aws (cost-filter-multiple-values) (-'x'-).oO(
(ins)> nice -n 19 make testacc TEST=./aws TESTARGS="-run=TestAccAWSBudgetsBudget_basic" && nice -n 19 make testac
c TEST=./aws TESTARGS="-run=TestAccAWSBudgetsBudget_prefix" && nice -n 19 make testacc TEST=./aws TESTARGS="-run=
TestAccAWSBudgetsBudget_notification"
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -parallel 20 -run=TestAccAWSBudgetsBudget_basic -timeout 120m
=== RUN   TestAccAWSBudgetsBudget_basic
=== PAUSE TestAccAWSBudgetsBudget_basic
=== CONT  TestAccAWSBudgetsBudget_basic
--- PASS: TestAccAWSBudgetsBudget_basic (39.27s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       39.297s
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -parallel 20 -run=TestAccAWSBudgetsBudget_prefix -timeout 120m
=== RUN   TestAccAWSBudgetsBudget_prefix
=== PAUSE TestAccAWSBudgetsBudget_prefix
=== CONT  TestAccAWSBudgetsBudget_prefix
--- PASS: TestAccAWSBudgetsBudget_prefix (34.90s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       34.924s
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -parallel 20 -run=TestAccAWSBudgetsBudget_notification -timeout 120m
=== RUN   TestAccAWSBudgetsBudget_notification
=== PAUSE TestAccAWSBudgetsBudget_notification
=== CONT  TestAccAWSBudgetsBudget_notification
--- PASS: TestAccAWSBudgetsBudget_notification (116.67s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       116.699s

@mozamimy mozamimy requested a review from a team June 22, 2019 06:09
@ghost ghost added size/L Managed by automation to categorize the size of a PR. documentation Introduces or discusses updates to documentation. service/budgets Issues and PRs that pertain to the budgets service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Jun 22, 2019
@mozamimy mozamimy changed the title Support multiple values per one filter in aws_budgets_budget resource Support multiple values per one cost filter in aws_budgets_budget resource Jun 22, 2019
@ButterflyServiceDesk
Copy link

@terraformbot can we get this in the next release please, pretty vital to be able to apply cost filters to multiple linked accounts

@iwt-gregorpoloczek
Copy link

Looking very much forward for the release of this feature! As of now, we cannot do our budget with terraform.

@aferg439
Copy link

aferg439 commented Apr 3, 2020

It's been almost a year now... Can this fix be implemented, please? Thanks!

@KevinGimbel
Copy link

Need this urgently!

Multi-tag budgets are pretty powerful. The original issue was opened in September 2018(!) - what is blocking this being merged and released?

@hummeltech
Copy link

Great work! What more needs to be done before this can be merged?

@yclian
Copy link

yclian commented May 10, 2020

In response to the "post"-COVID-19 economy, we are applying budget alerts extensively to many accounts in the organization, and some aws_budgets* resources are rather coarse and would only be possible if this PR is merged.

What else do we need to make this happen?

@hummeltech
Copy link

In case this helps, I've fixed the conflicts here and rebased on master:

master...hummeltech:cost-filter-multiple-values

@yashshah
Copy link

What is blocking this change? We would love to have this released

@cparmar
Copy link

cparmar commented Jan 5, 2021

This is also causing us some issues - any news on when this could be merged?

@ghost
Copy link

ghost commented Jan 8, 2021

I have also just run into this issue. I need to setup multiple cost_filters for data transfer usage.

Base automatically changed from master to main January 23, 2021 00:56
@breathingdust breathingdust requested a review from a team as a code owner January 23, 2021 00:56
@ChristophShyper
Copy link
Contributor

Can someone review this PR? It will be 2 years soon...

@jfortier-haptiq
Copy link

June 2021 and this is still an issue almost 2 years later! Can we get some eyes on this!

@chrono
Copy link

chrono commented Jun 4, 2021

Is there anything blocking this? Can anything be done by the community to get this [or any change making budgets work properly] merged?

We're now adding extra budget tags, increasing report data sizes and confusing future devs working with our AWS budget simply so that this can work with the IAAS/deployment tool [terraform] we're using.

@asosnovsky
Copy link

any updates on this, this ticket has been here for 2+ years and without it the budgets resource is somewhat pointless.

@elior379
Copy link

Any news about it?

We really need it !

mozamimy and others added 4 commits July 25, 2021 17:43
Now, we can specify multiple filter values per a cost filter key like

```
resource "aws_budgets_budget" "test" {
  # :
  # (snip)
  # :

  cost_filter {
    name = "Region"
    values = [
      "ap-northeast-1",
      "us-east-1",
    ]
  }

  cost_filter {
    name = "PurchaseType"
    values = [
      "Spot",
    ]
  }

  # :
  # (snip)
  # :
}
```

And we made `cost_filters` deprecated. That should be removed in future
releases.
ewbankkit and others added 9 commits July 25, 2021 17:43
Acceptance test output:

% make testacc TESTARGS='-run=TestAccAWSBudgetsBudgetAction_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSBudgetsBudgetAction_ -timeout 180m
=== RUN   TestAccAWSBudgetsBudgetAction_basic
=== PAUSE TestAccAWSBudgetsBudgetAction_basic
=== RUN   TestAccAWSBudgetsBudgetAction_disappears
=== PAUSE TestAccAWSBudgetsBudgetAction_disappears
=== CONT  TestAccAWSBudgetsBudgetAction_basic
=== CONT  TestAccAWSBudgetsBudgetAction_disappears
--- PASS: TestAccAWSBudgetsBudgetAction_disappears (29.50s)
--- PASS: TestAccAWSBudgetsBudgetAction_basic (29.58s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	32.689s
Acceptance test output:

% make testacc TESTARGS='-run=TestAccAWSBudgetsBudget_basic'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSBudgetsBudget_basic -timeout 180m
=== RUN   TestAccAWSBudgetsBudget_basic
=== PAUSE TestAccAWSBudgetsBudget_basic
=== CONT  TestAccAWSBudgetsBudget_basic
--- PASS: TestAccAWSBudgetsBudget_basic (23.28s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	26.567s
…e in ARN.

Acceptance test output:
% make testacc TESTARGS='-run=TestAccAWSBudgetsBudget_basic\|TestAccAWSBudgetsBudget_disappears\|TestAccAWSBudgetsBudgetAction_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSBudgetsBudget_basic\|TestAccAWSBudgetsBudget_disappears\|TestAccAWSBudgetsBudgetAction_ -timeout 180m
=== RUN   TestAccAWSBudgetsBudgetAction_basic
=== PAUSE TestAccAWSBudgetsBudgetAction_basic
=== RUN   TestAccAWSBudgetsBudgetAction_disappears
=== PAUSE TestAccAWSBudgetsBudgetAction_disappears
=== RUN   TestAccAWSBudgetsBudget_basic
=== PAUSE TestAccAWSBudgetsBudget_basic
=== RUN   TestAccAWSBudgetsBudget_disappears
=== PAUSE TestAccAWSBudgetsBudget_disappears
=== RUN   TestAccAWSBudgetsBudget_basicish
=== PAUSE TestAccAWSBudgetsBudget_basicish
=== CONT  TestAccAWSBudgetsBudgetAction_basic
=== CONT  TestAccAWSBudgetsBudget_disappears
=== CONT  TestAccAWSBudgetsBudget_basicish
=== CONT  TestAccAWSBudgetsBudget_basic
=== CONT  TestAccAWSBudgetsBudgetAction_disappears
--- PASS: TestAccAWSBudgetsBudget_disappears (10.17s)
--- PASS: TestAccAWSBudgetsBudget_basic (13.11s)
--- PASS: TestAccAWSBudgetsBudget_basicish (24.14s)
--- PASS: TestAccAWSBudgetsBudgetAction_disappears (24.38s)
--- PASS: TestAccAWSBudgetsBudgetAction_basic (58.92s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	62.207s
Acceptance test output:

% make testacc TESTARGS='-run=TestAccAWSBudgetsBudget_basic'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSBudgetsBudget_basic -timeout 180m
=== RUN   TestAccAWSBudgetsBudget_basic
=== PAUSE TestAccAWSBudgetsBudget_basic
=== RUN   TestAccAWSBudgetsBudget_basicish
=== PAUSE TestAccAWSBudgetsBudget_basicish
=== CONT  TestAccAWSBudgetsBudget_basic
=== CONT  TestAccAWSBudgetsBudget_basicish
--- PASS: TestAccAWSBudgetsBudget_basic (17.08s)
--- PASS: TestAccAWSBudgetsBudget_basicish (27.29s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	30.688s
Acceptance test output:

% make testacc TESTARGS='-run=TestAccAWSBudgetsBudget_CostTypes'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSBudgetsBudget_CostTypes -timeout 180m
=== RUN   TestAccAWSBudgetsBudget_CostTypes
=== PAUSE TestAccAWSBudgetsBudget_CostTypes
=== CONT  TestAccAWSBudgetsBudget_CostTypes
--- PASS: TestAccAWSBudgetsBudget_CostTypes (20.96s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	24.217s
Acceptance test output:

% make testacc TESTARGS='-run=TestAccAWSBudgetsBudget_Notifications'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSBudgetsBudget_Notifications -timeout 180m
=== RUN   TestAccAWSBudgetsBudget_Notifications
=== PAUSE TestAccAWSBudgetsBudget_Notifications
=== CONT  TestAccAWSBudgetsBudget_Notifications
--- PASS: TestAccAWSBudgetsBudget_Notifications (27.42s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	30.688s
@ewbankkit ewbankkit force-pushed the cost-filter-multiple-values branch from 69c93d9 to 88c52a6 Compare July 27, 2021 14:35
@github-actions github-actions bot added size/XL Managed by automation to categorize the size of a PR. and removed size/L Managed by automation to categorize the size of a PR. labels Jul 27, 2021
Copy link
Contributor

@ewbankkit ewbankkit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀.

Commercial
% make testacc TESTARGS='-run=TestAccAWSBudgetsBudgetAction_\|TestAccAWSBudgetsBudget_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSBudgetsBudgetAction_\|TestAccAWSBudgetsBudget_ -timeout 180m
=== RUN   TestAccAWSBudgetsBudgetAction_basic
=== PAUSE TestAccAWSBudgetsBudgetAction_basic
=== RUN   TestAccAWSBudgetsBudgetAction_disappears
=== PAUSE TestAccAWSBudgetsBudgetAction_disappears
=== RUN   TestAccAWSBudgetsBudget_basic
=== PAUSE TestAccAWSBudgetsBudget_basic
=== RUN   TestAccAWSBudgetsBudget_Name_Generated
=== PAUSE TestAccAWSBudgetsBudget_Name_Generated
=== RUN   TestAccAWSBudgetsBudget_NamePrefix
=== PAUSE TestAccAWSBudgetsBudget_NamePrefix
=== RUN   TestAccAWSBudgetsBudget_disappears
=== PAUSE TestAccAWSBudgetsBudget_disappears
=== RUN   TestAccAWSBudgetsBudget_CostTypes
=== PAUSE TestAccAWSBudgetsBudget_CostTypes
=== RUN   TestAccAWSBudgetsBudget_Notifications
=== PAUSE TestAccAWSBudgetsBudget_Notifications
=== CONT  TestAccAWSBudgetsBudgetAction_basic
=== CONT  TestAccAWSBudgetsBudget_disappears
=== CONT  TestAccAWSBudgetsBudget_NamePrefix
=== CONT  TestAccAWSBudgetsBudget_basic
=== CONT  TestAccAWSBudgetsBudget_Notifications
=== CONT  TestAccAWSBudgetsBudget_Name_Generated
=== CONT  TestAccAWSBudgetsBudgetAction_disappears
=== CONT  TestAccAWSBudgetsBudget_CostTypes
--- PASS: TestAccAWSBudgetsBudget_disappears (16.47s)
--- PASS: TestAccAWSBudgetsBudget_NamePrefix (20.88s)
--- PASS: TestAccAWSBudgetsBudget_Name_Generated (20.94s)
--- PASS: TestAccAWSBudgetsBudget_basic (21.07s)
--- PASS: TestAccAWSBudgetsBudget_CostTypes (29.39s)
--- PASS: TestAccAWSBudgetsBudgetAction_basic (29.42s)
--- PASS: TestAccAWSBudgetsBudget_Notifications (33.87s)
--- PASS: TestAccAWSBudgetsBudgetAction_disappears (60.02s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	63.512s
GovCloud
% make testacc TESTARGS='-run=TestAccAWSBudgetsBudgetAction_\|TestAccAWSBudgetsBudget_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSBudgetsBudgetAction_\|TestAccAWSBudgetsBudget_ -timeout 180m
=== RUN   TestAccAWSBudgetsBudgetAction_basic
=== PAUSE TestAccAWSBudgetsBudgetAction_basic
=== RUN   TestAccAWSBudgetsBudgetAction_disappears
=== PAUSE TestAccAWSBudgetsBudgetAction_disappears
=== RUN   TestAccAWSBudgetsBudget_basic
=== PAUSE TestAccAWSBudgetsBudget_basic
=== RUN   TestAccAWSBudgetsBudget_Name_Generated
=== PAUSE TestAccAWSBudgetsBudget_Name_Generated
=== RUN   TestAccAWSBudgetsBudget_NamePrefix
=== PAUSE TestAccAWSBudgetsBudget_NamePrefix
=== RUN   TestAccAWSBudgetsBudget_disappears
=== PAUSE TestAccAWSBudgetsBudget_disappears
=== RUN   TestAccAWSBudgetsBudget_CostTypes
=== PAUSE TestAccAWSBudgetsBudget_CostTypes
=== RUN   TestAccAWSBudgetsBudget_Notifications
=== PAUSE TestAccAWSBudgetsBudget_Notifications
=== CONT  TestAccAWSBudgetsBudgetAction_basic
=== CONT  TestAccAWSBudgetsBudget_disappears
=== CONT  TestAccAWSBudgetsBudget_Notifications
=== CONT  TestAccAWSBudgetsBudget_CostTypes
=== CONT  TestAccAWSBudgetsBudgetAction_disappears
=== CONT  TestAccAWSBudgetsBudget_Name_Generated
=== CONT  TestAccAWSBudgetsBudget_basic
=== CONT  TestAccAWSBudgetsBudget_NamePrefix
=== CONT  TestAccAWSBudgetsBudgetAction_basic
    provider_test.go:735: skipping tests; partition aws-us-gov does not support budgets service
=== CONT  TestAccAWSBudgetsBudgetAction_disappears
    provider_test.go:735: skipping tests; partition aws-us-gov does not support budgets service
=== CONT  TestAccAWSBudgetsBudget_disappears
    provider_test.go:735: skipping tests; partition aws-us-gov does not support budgets service
--- SKIP: TestAccAWSBudgetsBudgetAction_basic (1.46s)
--- SKIP: TestAccAWSBudgetsBudget_disappears (1.46s)
--- SKIP: TestAccAWSBudgetsBudgetAction_disappears (1.46s)
=== CONT  TestAccAWSBudgetsBudget_basic
    provider_test.go:735: skipping tests; partition aws-us-gov does not support budgets service
--- SKIP: TestAccAWSBudgetsBudget_basic (1.46s)
=== CONT  TestAccAWSBudgetsBudget_Name_Generated
    provider_test.go:735: skipping tests; partition aws-us-gov does not support budgets service
--- SKIP: TestAccAWSBudgetsBudget_Name_Generated (1.46s)
=== CONT  TestAccAWSBudgetsBudget_Notifications
    provider_test.go:735: skipping tests; partition aws-us-gov does not support budgets service
=== CONT  TestAccAWSBudgetsBudget_NamePrefix
    provider_test.go:735: skipping tests; partition aws-us-gov does not support budgets service
--- SKIP: TestAccAWSBudgetsBudget_Notifications (1.46s)
=== CONT  TestAccAWSBudgetsBudget_CostTypes
    provider_test.go:735: skipping tests; partition aws-us-gov does not support budgets service
--- SKIP: TestAccAWSBudgetsBudget_NamePrefix (1.46s)
--- SKIP: TestAccAWSBudgetsBudget_CostTypes (1.46s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	4.404s

@ewbankkit
Copy link
Contributor

@mozamimy Thanks for the contribution 🎉 👏.
As we can only deprecate arguments at a major version I made changes to note that cost_filter is now preferred over cost_filters and added #20330 to deprecate cost_filters with v4.0.0 which should be available later this year.

@ewbankkit ewbankkit merged commit f5a1bb5 into hashicorp:main Jul 27, 2021
@github-actions github-actions bot added this to the v3.52.0 milestone Jul 27, 2021
@mozamimy mozamimy deleted the cost-filter-multiple-values branch July 29, 2021 00:06
@github-actions
Copy link

This functionality has been released in v3.52.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@ChristophShyper
Copy link
Contributor

ChristophShyper commented Jul 30, 2021

When using

(...)
    cost_filter = {
      name   = "LinkedAccount"
      values = local.linked_accounts
    }
(...)

I'm getting

╷
│ Error: Unsupported argument
│
│   on ../submodule-costs/main.tf line 127, in resource "aws_budgets_budget" "this":
│  127:   cost_filter = {
│
│ An argument named "cost_filter" is not expected here. Did you mean
│ "cost_filters"?
╵
ERRO[0052] 1 error occurred:
	* exit status 1

Using newest provider version

- Installing hashicorp/aws v3.52.0...
- Installed hashicorp/aws v3.52.0 (signed by HashiCorp)

When I rename cost_filter to cost_filters I'm getting

╷
│ Error: error creating Budget (Monthly budget): InvalidParameterException: Unable to create/update budget - values is not in the supported in cost budget dimension set: [PurchaseType, UsageTypeGroup, Service, Operation, UsageType, BillingEntity, CostCategory, LinkedAccount, TagKeyValue, AZ, Region, InstanceType]
│
│   with module.costs.aws_budgets_budget.this,
│   on ../submodule-costs/main.tf line 120, in resource "aws_budgets_budget" "this":
│  120: resource "aws_budgets_budget" "this" {
│
╵

@dmrzzz
Copy link

dmrzzz commented Aug 2, 2021

@ChristophShyper use cost_filter { instead of cost_filter = {

@ChristophShyper
Copy link
Contributor

You're right! My bad, it changed type :)

@tammyllf
Copy link

tammyllf commented Aug 5, 2021

   cost_filter = {
       name   = "LinkedAccount"
       values = local.link_account_id
       }

Amend my cost_filter to the above, managed to run terraform init.
But hit into the following error when try to run terraform plan

Error: Unsupported block type

  on purple.tf line 35, in resource "aws_budgets_budget" "purple":
  35:   cost_filter {

Blocks of type "cost_filter" are not expected here.

I have multiple linked account, and multiple cost_filter within one tf file.
Can someone advise, what is the problem?

Thanks

@github-actions
Copy link

github-actions bot commented Sep 5, 2021

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 5, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Introduces or discusses updates to documentation. service/budgets Issues and PRs that pertain to the budgets service. size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet