Skip to content

Commit

Permalink
Add support for IBM Cloud Billing Converter (#360)
Browse files Browse the repository at this point in the history 
Signed-off-by: Chris Waddington <104161708+chrisw-ibm@users.noreply.github.com>
  • Loading branch information
@chrisw-ibm
chrisw-ibm authored Aug 30, 2024
1 parent 68de059 commit d97ceb0
Show file tree
Hide file tree
Showing 37 changed files with 489 additions and 4 deletions.
49 changes: 49 additions & 0 deletions conversion_rules_export/csv/ibm.csv
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
FOCUS Dimension,Transform Step,Source Column,Source Column Type,Transform Type,Filters/Process/Etc.
AvailabilityZone,0,Not Defined,Not Defined,Not Defined,Not Defined
ChargeDescription,0,Not Defined,Not Defined,Not Defined,Not Defined
CommitmentDiscountCategory,0,Not Defined,Not Defined,Not Defined,Not Defined
CommitmentDiscountName,0,Not Defined,Not Defined,Not Defined,Not Defined
CommitmentDiscountType,0,Not Defined,Not Defined,Not Defined,Not Defined
ConsumedQuantity,0,Not Defined,Not Defined,Not Defined,Not Defined
ConsumedUnit,0,Not Defined,Not Defined,Not Defined,Not Defined
EffectiveCost,0,Not Defined,Not Defined,Not Defined,Not Defined
ListUnitPrice,0,Not Defined,Not Defined,Not Defined,Not Defined
SkuId,0,Not Defined,Not Defined,Not Defined,Not Defined
SkuPriceId,0,Not Defined,Not Defined,Not Defined,Not Defined
BilledCost,1,Volume Cost,Not Defined,RENAME_COLUMN,
BillingAccountId,1,Account Owner ID,Not Defined,RENAME_COLUMN,
BillingAccountName,1,Account Name,Not Defined,RENAME_COLUMN,
BillingCurrency,1,Currency,Not Defined,RENAME_COLUMN,
BillingPeriodEnd,1,BillingPeriodStart,Not Defined,MONTH_END,
BillingPeriodStart,1,Billing Month,Not Defined,PARSE_DATETIME,%Y-%m
BillingPeriodStart,2,BillingPeriodStart,Not Defined,ASSIGN_UTC_TIMEZONE,
ChargeCategory,1,NA,Not Defined,ASSIGN_STATIC_VALUE,"static_value: Usage
"
ChargeFrequency,1,NA,Not Defined,ASSIGN_STATIC_VALUE,"static_value: Usage-Based
"
ChargePeriodEnd,1,Created Time,Not Defined,RENAME_COLUMN,
ChargePeriodStart,1,Billing Month,Not Defined,PARSE_DATETIME,%Y-%m
ChargePeriodStart,2,ChargePeriodStart,Not Defined,ASSIGN_UTC_TIMEZONE,
CommitmentDiscountId,1,Discount ID,Not Defined,RENAME_COLUMN,
InvoiceIssuer,1,NA,Not Defined,ASSIGN_STATIC_VALUE,"static_value: IBM Cloud
"
ListCost,1,Original Cost,Not Defined,RENAME_COLUMN,
PricingCategory,1,NA,Not Defined,ASSIGN_STATIC_VALUE,"static_value: Standard
"
PricingQuantity,1,Usage Quantity,Not Defined,RENAME_COLUMN,
PricingUnit,1,Usage Unit,Not Defined,RENAME_COLUMN,
Provider,1,NA,Not Defined,ASSIGN_STATIC_VALUE,"static_value: IBM Cloud
"
Publisher,1,NA,Not Defined,ASSIGN_STATIC_VALUE,"static_value: IBM Cloud
"
RegionId,1,Region,Not Defined,RENAME_COLUMN,
ResourceId,1,Instance ID,Not Defined,RENAME_COLUMN,
ResourceName,1,Instance Name,Not Defined,RENAME_COLUMN,
ResourceType,1,ResourceId,Not Defined,STRING_FUNCTIONS,"{'steps': [{'operation_type': 'split', 'split_by': ':', 'index': 4}]}"
ServiceCategory,1,ResourceType,Not Defined,LOOKUP,"destination_value: ServiceCategory
reference_dataset_path: conversion_configs/ibm/mapping_files/ibm_category_mapping.csv
source_value: ResourceType
"
ServiceName,1,Service Name,Not Defined,RENAME_COLUMN,
SubAccountId,1,Account Owner ID,Not Defined,RENAME_COLUMN,
SubAccountName,1,Account Name,Not Defined,RENAME_COLUMN,
43 changes: 43 additions & 0 deletions conversion_rules_export/markdown/ibm.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
| FOCUS Dimension | Transform Step | Source Column | Source Column Type | Transform Type | Filters/Process/Etc. |
|:---------------------------|-----------------:|:-------------------|:---------------------|:--------------------|:--------------------------------------------------------------------------------------|
| AvailabilityZone | 0 | Not Defined | Not Defined | Not Defined | Not Defined |
| ChargeDescription | 0 | Not Defined | Not Defined | Not Defined | Not Defined |
| CommitmentDiscountCategory | 0 | Not Defined | Not Defined | Not Defined | Not Defined |
| CommitmentDiscountName | 0 | Not Defined | Not Defined | Not Defined | Not Defined |
| CommitmentDiscountType | 0 | Not Defined | Not Defined | Not Defined | Not Defined |
| ConsumedQuantity | 0 | Not Defined | Not Defined | Not Defined | Not Defined |
| ConsumedUnit | 0 | Not Defined | Not Defined | Not Defined | Not Defined |
| EffectiveCost | 0 | Not Defined | Not Defined | Not Defined | Not Defined |
| ListUnitPrice | 0 | Not Defined | Not Defined | Not Defined | Not Defined |
| SkuId | 0 | Not Defined | Not Defined | Not Defined | Not Defined |
| SkuPriceId | 0 | Not Defined | Not Defined | Not Defined | Not Defined |
| BilledCost | 1 | Volume Cost | Not Defined | RENAME_COLUMN | |
| BillingAccountId | 1 | Account Owner ID | Not Defined | RENAME_COLUMN | |
| BillingAccountName | 1 | Account Name | Not Defined | RENAME_COLUMN | |
| BillingCurrency | 1 | Currency | Not Defined | RENAME_COLUMN | |
| BillingPeriodEnd | 1 | BillingPeriodStart | Not Defined | MONTH_END | |
| BillingPeriodStart | 1 | Billing Month | Not Defined | PARSE_DATETIME | %Y-%m |
| BillingPeriodStart | 2 | BillingPeriodStart | Not Defined | ASSIGN_UTC_TIMEZONE | |
| ChargeCategory | 1 | NA | Not Defined | ASSIGN_STATIC_VALUE | static_value: Usage |
| ChargeFrequency | 1 | NA | Not Defined | ASSIGN_STATIC_VALUE | static_value: Usage-Based |
| ChargePeriodEnd | 1 | Created Time | Not Defined | RENAME_COLUMN | |
| ChargePeriodStart | 1 | Billing Month | Not Defined | PARSE_DATETIME | %Y-%m |
| ChargePeriodStart | 2 | ChargePeriodStart | Not Defined | ASSIGN_UTC_TIMEZONE | |
| CommitmentDiscountId | 1 | Discount ID | Not Defined | RENAME_COLUMN | |
| InvoiceIssuer | 1 | NA | Not Defined | ASSIGN_STATIC_VALUE | static_value: IBM Cloud |
| ListCost | 1 | Original Cost | Not Defined | RENAME_COLUMN | |
| PricingCategory | 1 | NA | Not Defined | ASSIGN_STATIC_VALUE | static_value: Standard |
| PricingQuantity | 1 | Usage Quantity | Not Defined | RENAME_COLUMN | |
| PricingUnit | 1 | Usage Unit | Not Defined | RENAME_COLUMN | |
| Provider | 1 | NA | Not Defined | ASSIGN_STATIC_VALUE | static_value: IBM Cloud |
| Publisher | 1 | NA | Not Defined | ASSIGN_STATIC_VALUE | static_value: IBM Cloud |
| RegionId | 1 | Region | Not Defined | RENAME_COLUMN | |
| ResourceId | 1 | Instance ID | Not Defined | RENAME_COLUMN | |
| ResourceName | 1 | Instance Name | Not Defined | RENAME_COLUMN | |
| ResourceType | 1 | ResourceId | Not Defined | STRING_FUNCTIONS | {'steps': [{'operation_type': 'split', 'split_by': ':', 'index': 4}]} |
| ServiceCategory | 1 | ResourceType | Not Defined | LOOKUP | destination_value: ServiceCategory |
| | | | | | reference_dataset_path: conversion_configs/ibm/mapping_files/ibm_category_mapping.csv |
| | | | | | source_value: ResourceType |
| ServiceName | 1 | Service Name | Not Defined | RENAME_COLUMN | |
| SubAccountId | 1 | Account Owner ID | Not Defined | RENAME_COLUMN | |
| SubAccountName | 1 | Account Name | Not Defined | RENAME_COLUMN | |
5 changes: 4 additions & 1 deletion focus_converter_base/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# FOCUS Converter

The FOCUS Converter is a command-line utility to convert billing data files from popular public cloud providers,
such as **Amazon Web Services**, **Microsoft Azure**, **Google Cloud** and **Oracle Cloud**, into the common
such as **Amazon Web Services**, **Microsoft Azure**, **Google Cloud**, **IBM Cloud** and **Oracle Cloud**, into the common
schema known as FOCUS. You can read the specification at [FinOps-Open-Cost-and-Usage-Spec/FOCUS_Spec].

The converter is optimized for:
Expand All @@ -16,6 +16,7 @@ The converter is optimized for:
* [Amazon Web Services]
* [Google Cloud]
* [Microsoft Azure]
* [IBM Cloud]
* [Oracle Cloud]

Want to add your own? See [CONTRIBUTING.md]
Expand Down Expand Up @@ -89,6 +90,8 @@ We're excited to work together. Please see [CONTRIBUTING.md] for information on

[Microsoft Azure]: https://github.com/finopsfoundation/focus_converters/tree/master/focus_converter_base/focus_converter/conversion_configs/azure

[IBM Cloud]: https://github.com/finopsfoundation/focus_converters/tree/master/focus_converter_base/focus_converter/conversion_configs/ibm

[Oracle Cloud]: https://github.com/finopsfoundation/focus_converters/tree/master/focus_converter_base/focus_converter/conversion_configs/oci

[Rules Export Markdown]: https://github.com/finopsfoundation/focus_converters/tree/master/conversion_rules_export/markdown
Expand Down
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_billed_cost_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Volume Cost to BilledCost
conversion_type: rename_column
column: Volume Cost
focus_column: BilledCost
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_billing_account_id_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Account Owner ID to BillingAccountId
conversion_type: rename_column
column: Account Owner ID
focus_column: BillingAccountId
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_billing_account_name_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Account Name to BillingAccountName
conversion_type: rename_column
column: Account Name
focus_column: BillingAccountName
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_billing_currency_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Currency to BillingCurrency
conversion_type: rename_column
column: Currency
focus_column: BillingCurrency
5 changes: 5 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_billing_period_start_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
plan_name: parse YYYY-mm from Billing Month
conversion_type: parse_datetime
column: Billing Month
focus_column: BillingPeriodStart
conversion_args: "%Y-%m"
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_billing_period_start_S002.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: set BillingPeriodStart as UTC
conversion_type: assign_utc_timezone
column: BillingPeriodStart
focus_column: BillingPeriodStart
6 changes: 6 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_charge_category_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
plan_name: Assign ChargeCategory
conversion_type: static_value
conversion_args:
static_value: Usage
column: NA
focus_column: ChargeCategory
6 changes: 6 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_charge_frequency_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
plan_name: Assign ChargeFrequency
conversion_type: static_value
conversion_args:
static_value: Usage-Based
column: NA
focus_column: ChargeFrequency
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_charge_period_end_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Created Time for ChargePeriodEnd
conversion_type: rename_column
column: Created Time
focus_column: ChargePeriodEnd
5 changes: 5 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_charge_period_start_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
plan_name: parse YYYY-mm from Billing Month for ChargePeriodStart
conversion_type: parse_datetime
column: Billing Month
focus_column: ChargePeriodStart
conversion_args: "%Y-%m"
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_charge_period_start_S002.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: set ChargePeriodStart as UTC
conversion_type: assign_utc_timezone
column: ChargePeriodStart
focus_column: ChargePeriodStart
4 changes: 4 additions & 0 deletions ..._converter_base/focus_converter/conversion_configs/ibm/1_commitment_discount_id_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Discount ID to CommitmentDiscountID
conversion_type: rename_column
column: Discount ID
focus_column: CommitmentDiscountId
7 changes: 7 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_invoice_issuer_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Use a lookup for third parties
plan_name: Assign InvoiceIssuer
conversion_type: static_value
conversion_args:
static_value: IBM Cloud
column: NA
focus_column: InvoiceIssuer
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_list_cost_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Original Cost to ListCost
conversion_type: rename_column
column: Original Cost
focus_column: ListCost
6 changes: 6 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_pricing_category_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
plan_name: Assign PricingCategory
conversion_type: static_value
conversion_args:
static_value: Standard
column: NA
focus_column: PricingCategory
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_pricing_quantity_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Usage Quantity to PricingQuantity
conversion_type: rename_column
column: Usage Quantity
focus_column: PricingQuantity
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_pricing_unit_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Usage Unit to PricingUnit
conversion_type: rename_column
column: Usage Unit
focus_column: PricingUnit
6 changes: 6 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_provider_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
plan_name: Assign provider
conversion_type: static_value
conversion_args:
static_value: IBM Cloud
column: NA
focus_column: Provider
6 changes: 6 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_publisher_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
plan_name: Assign Publisher
conversion_type: static_value
conversion_args:
static_value: IBM Cloud
column: NA
focus_column: Publisher
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_region_id_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Instance ID to ResourceID
conversion_type: rename_column
column: Region
focus_column: RegionId
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_resource_id_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Instance ID to ResourceID
conversion_type: rename_column
column: Instance ID
focus_column: ResourceId
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_resource_name_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Instance Name to ResourceName
conversion_type: rename_column
column: Instance Name
focus_column: ResourceName
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_service_name_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: Rename Service Name to ServiceName
conversion_type: rename_column
column: Service Name
focus_column: ServiceName
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_sub_account_id_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Account Owner ID to SubAccountID
conversion_type: rename_column
column: Account Owner ID
focus_column: SubAccountId
4 changes: 4 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/1_sub_account_name_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
plan_name: rename Account Name to SubAccountName
conversion_type: rename_column
column: Account Name
focus_column: SubAccountName
5 changes: 5 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/2_billing_period_end_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
plan_name: compute BillingPeriodEnd
conversion_type: month_end
column: BillingPeriodStart
focus_column: BillingPeriodEnd

9 changes: 9 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/2_resource_type_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
plan_name: Get resource type from resource Id
conversion_type: string_functions
column: ResourceId
focus_column: ResourceType
conversion_args:
steps:
- operation_type: split
split_by: ":"
index: 4
8 changes: 8 additions & 0 deletions focus_converter_base/focus_converter/conversion_configs/ibm/3_service_category_S001.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
plan_name: Generate ServiceCategory for ibm data using a lookup.
column: ResourceType
conversion_type: lookup
focus_column: ServiceCategory
conversion_args:
reference_dataset_path: "conversion_configs/ibm/mapping_files/ibm_category_mapping.csv"
source_value: ResourceType
destination_value: ServiceCategory
Loading

0 comments on commit d97ceb0

Please sign in to comment.