Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding support for --includeTags option #108

Merged
merged 1 commit into from
Feb 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ OPTIONS:
--useUSD Force the use of USD for the currency. Defaults to false to use the currency returned by the API
--skipHeader Skip header creation for specific output formats. Useful when appending the output from multiple runs into one file. Defaults to false
--filter Filter the output by the specified properties. Defaults to no filtering and can be multiple values.
--includeTags Include Tags from the selected dimension. Valid only for DailyCost report and output to Json, JsonC or Csv. Ignored in the rest of reports and output formats.
-m, --metric ActualCost The metric to use for the costs. Defaults to ActualCost. (ActualCost, AmortizedCost)

COMMANDS:
Expand Down Expand Up @@ -207,6 +208,22 @@ The above screenshots show the default console output, but the other formatters

The available dimensions are: `ResourceGroup`,`ResourceGroupName`,`ResourceLocation`,`ConsumedService`,`ResourceType`,`ResourceId`,`MeterId`,`BillingMonth`,`MeterCategory`,`MeterSubcategory`,`Meter`,`AccountName`,`DepartmentName`,`SubscriptionId`,`SubscriptionName`,`ServiceName`,`ServiceTier`,`EnrollmentAccountName`,`BillingAccountId`,`ResourceGuid`,`BillingPeriod`,`InvoiceNumber`,`ChargeType`,`PublisherType`,`ReservationId`,`ReservationName`,`Frequency`,`PartNumber`,`CostAllocationRuleName`,`MarkupRuleName`,`PricingModel`,`BenefitId`,`BenefitName`

### Include Tags
This option allows to include the dimensions' Tags in the same row. Tags allow cost analysis customization. Adding the Tags from the dimension allows complementary analysis in tools like Power BI. This option is enabled for DailyCost report and for Json, JsonC, and Csv expor formats. Using other formats, ignores the option.

The following query shows the daily costs for subscription x group by resource group name including the tags for the resource group ready to export to Csv:

```bash
azure-cost dailyCosts -s XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX --dimension ResourceGroupName --includeTags -o Csv
```

That would extend into a column called Tags the resource group tags in Json format:
```bash
[""\""cost-center\"":\""my_cost_center\"""",""\""owner\"":\""my_email@email.com\""""]
```
Note that the Json column should be parsed in the analytical tool.


### Detect Anomalies

Based on the daily cost data, this command will try to detect anomalies and trends. It will scan for the following anomalies:
Expand Down
7 changes: 6 additions & 1 deletion src/Commands/CostSettings.cs
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,12 @@ public class CostSettings : LogCommandSettings, ICostSettings
[Description("The metric to use for the costs. Defaults to ActualCost. (ActualCost, AmortizedCost)")]
[DefaultValue(MetricType.ActualCost)]
public MetricType Metric { get; set; } = MetricType.ActualCost;


[CommandOption("--includeTags")]
[Description("Include Tags from the selected dimension. The option is used for DailyCost report and output to Json, JsonC or Csv. Valid only for DailyCost report and output to Json, JsonC or Csv. Ignored in other reports and output formats.")]
[DefaultValue(false)]
public bool IncludeTags { get; set; }


public Scope GetScope
{
Expand Down
12 changes: 10 additions & 2 deletions src/Commands/DailyCost/DailyCost.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
using AzureCostCli.OutputFormatters;
using Spectre.Console;
using Spectre.Console.Cli;
using System;

namespace AzureCostCli.Commands.DailyCost;

Expand Down Expand Up @@ -86,6 +87,12 @@ public override async Task<int> ExecuteAsync(CommandContext context, DailyCostSe

IEnumerable<CostDailyItem> dailyCost = new List<CostDailyItem>();

// if output format is not csv, json, or jsonc, then don't include tags
if (settings.Output.ToString().ToLower() != "json" && settings.Output.ToString().ToLower() != "jsonc" && settings.Output.ToString().ToLower() != "csv")
{
settings.IncludeTags = false;
}

await AnsiConsoleExt.Status()
.StartAsync("Fetching daily cost data...", async ctx =>
{
Expand All @@ -96,12 +103,13 @@ await AnsiConsoleExt.Status()
settings.Metric,
settings.Dimension,
settings.Timeframe,
settings.From, settings.To);
settings.From, settings.To,
settings.IncludeTags);
});

// Write the output
await _outputFormatters[settings.Output]
.WriteDailyCost(settings, dailyCost);
.WriteDailyCost(settings, dailyCost);

return 0; // Omitted
}
Expand Down
3 changes: 2 additions & 1 deletion src/Commands/DetectAnomaly/DetectAnomaly.cs
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,8 @@ public override async Task<int> ExecuteAsync(CommandContext context, DetectAnoma
settings.Metric,
settings.Dimension,
settings.Timeframe,
settings.From, settings.To);
settings.From, settings.To,
false);

var costAnalyzer = new CostAnalyzer(settings);

Expand Down
35 changes: 23 additions & 12 deletions src/CostApi/AzureCostApiRetriever.cs
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ public async Task<IEnumerable<CostItem>> RetrieveCosts(bool includeDebugOutput,
TimeframeType timeFrame, DateOnly from, DateOnly to)
{
var filters = GenerateFilters(filter);
var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2021-10-01&$top=5000");
var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2023-03-01&$top=5000");

var payload = new
{
Expand Down Expand Up @@ -210,7 +210,7 @@ public async Task<IEnumerable<CostItem>> RetrieveCosts(bool includeDebugOutput,

var currency = row[3].ToString();

var costItem = new CostItem(date, value, valueUsd, currency);
var costItem = new CostItem(date, value, valueUsd, currency, "");
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You set the currency to an empty string here. A specific reason for that? This should represent the currency the billing is in.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. Last argument is the Tag. Reason to be empty is because in that method includeTags do not apply

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh sorry, misreading it here.

items.Add(costItem);
}

Expand All @@ -222,7 +222,7 @@ public async Task<IEnumerable<CostItem>> RetrieveCosts(bool includeDebugOutput,
public async Task<IEnumerable<CostNamedItem>> RetrieveCostByServiceName(bool includeDebugOutput,
Scope scope, string[] filter, MetricType metric, TimeframeType timeFrame, DateOnly from, DateOnly to)
{
var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2021-10-01&$top=5000");
var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2023-03-01&$top=5000");

var payload = new
{
Expand Down Expand Up @@ -294,7 +294,7 @@ public async Task<IEnumerable<CostNamedItem>> RetrieveCostByLocation(bool includ
string[] filter,MetricType metric,
TimeframeType timeFrame, DateOnly from, DateOnly to)
{
var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2021-10-01&$top=5000");
var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2023-03-01&$top=5000");

var payload = new
{
Expand Down Expand Up @@ -366,7 +366,7 @@ public async Task<IEnumerable<CostNamedItem>> RetrieveCostByResourceGroup(bool i
Scope scope, string[] filter,MetricType metric,
TimeframeType timeFrame, DateOnly from, DateOnly to)
{
var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2021-10-01&$top=5000");
var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2023-03-01&$top=5000");

var payload = new
{
Expand Down Expand Up @@ -443,7 +443,7 @@ public async Task<IEnumerable<CostNamedItem>> RetrieveCostBySubscription(bool in
Scope scope, string[] filter, MetricType metric,
TimeframeType timeFrame, DateOnly from, DateOnly to)
{
var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2021-10-01&$top=5000");
var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2023-03-01&$top=5000");

var payload = new
{
Expand Down Expand Up @@ -518,10 +518,9 @@ public async Task<IEnumerable<CostNamedItem>> RetrieveCostBySubscription(bool in

public async Task<IEnumerable<CostDailyItem>> RetrieveDailyCost(bool includeDebugOutput,
Scope scope, string[] filter, MetricType metric, string dimension,
TimeframeType timeFrame, DateOnly from, DateOnly to)
TimeframeType timeFrame, DateOnly from, DateOnly to, bool includeTags)
{
var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2021-10-01&$top=5000");

var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2023-03-01&$top=5000");

var payload = new
{
Expand All @@ -537,6 +536,7 @@ public async Task<IEnumerable<CostDailyItem>> RetrieveDailyCost(bool includeDebu
dataSet = new
{
granularity = "Daily",
include = includeTags ? new[] { "Tags" } : null,
aggregation = new
{
totalCost = new
Expand Down Expand Up @@ -587,9 +587,20 @@ public async Task<IEnumerable<CostDailyItem>> RetrieveDailyCost(bool includeDebu
var value = double.Parse(row[0].ToString(), CultureInfo.InvariantCulture);
var valueUsd = double.Parse(row[1].ToString(), CultureInfo.InvariantCulture);

// if includeTags is true, row[5] is the tag, and row[6] is the currency, otherwise row[5] is the currency
var currency = row[5].ToString();
var tags = "";

// if includeTags is true, switch the value between currency and tags
// that's the order how the API REST exposes the resultset
if (includeTags)
{
System.Text.Json.JsonElement element = row[5];
tags = element.GetRawText();
currency = row[6].ToString();
}

var costItem = new CostDailyItem(date, resourceGroupName, value, valueUsd, currency);
var costItem = new CostDailyItem(date, resourceGroupName, value, valueUsd, currency, tags);
items.Add(costItem);
}

Expand Down Expand Up @@ -674,7 +685,7 @@ public async Task<IEnumerable<CostItem>> RetrieveForecastedCosts(bool includeDeb

var currency = row[3].ToString();

var costItem = new CostItem(date, value, value, currency);
var costItem = new CostItem(date, value, value, currency, "");
items.Add(costItem);
}
}
Expand All @@ -696,7 +707,7 @@ public async Task<IEnumerable<CostResourceItem>> RetrieveCostForResources(bool i
DateOnly from,
DateOnly to)
{
var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2021-10-01&$top=5000");
var uri = DeterminePath(scope, "/providers/Microsoft.CostManagement/query?api-version=2023-03-01&$top=5000");

object grouping;
if (excludeMeterDetails == false)
Expand Down
3 changes: 2 additions & 1 deletion src/CostApi/CostDailyItem.cs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
namespace AzureCostCli.CostApi;

public record CostDailyItem(DateOnly Date, string Name, double Cost, double CostUsd, string Currency);
public record CostDailyItem(DateOnly Date, string Name, double Cost, double CostUsd, string Currency, string Tags);
public record CostDailyItemWithoutTags(DateOnly Date, string Name, double Cost, double CostUsd, string Currency);
2 changes: 1 addition & 1 deletion src/CostApi/CostItem.cs
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
namespace AzureCostCli.CostApi;

public record CostItem(DateOnly Date, double Cost, double CostUsd, string Currency);
public record CostItem(DateOnly Date, double Cost, double CostUsd, string Currency, string Tags);
2 changes: 1 addition & 1 deletion src/CostApi/ICostRetriever.cs
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Task<IEnumerable<CostResourceItem>> RetrieveCostForResources(bool settingsDebug,
Task<IEnumerable<UsageDetails>> RetrieveUsageDetails(bool includeDebugOutput,
Scope scope, string filter, DateOnly from, DateOnly to);

Task<IEnumerable<CostDailyItem>> RetrieveDailyCost(bool settingsDebug, Scope scope, string[] filter,MetricType metric, string dimension, TimeframeType settingsTimeframe, DateOnly settingsFrom, DateOnly settingsTo);
Task<IEnumerable<CostDailyItem>> RetrieveDailyCost(bool settingsDebug, Scope scope, string[] filter,MetricType metric, string dimension, TimeframeType settingsTimeframe, DateOnly settingsFrom, DateOnly settingsTo, bool includeTags);
}


Expand Down
23 changes: 22 additions & 1 deletion src/OutputFormatters/CsvOutputFormatter.cs
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,28 @@

public override Task WriteDailyCost(DailyCostSettings settings, IEnumerable<CostDailyItem> dailyCosts)
{
return ExportToCsv(settings.SkipHeader, dailyCosts);
// code to create the column Tags only when needed
// small trick with the records dailyCostItemWithoutTags, and dailyCostItem
if (settings.IncludeTags == false)
{
var dailyCostsWithoutTags = new List<CostDailyItemWithoutTags>();
foreach (var item in dailyCosts)
{
var newItem = new CostDailyItemWithoutTags(
Name: item.Name,
Date: item.Date,
Cost: item.Cost,
Currency: item.Currency,
CostUsd: item.CostUsd
);
dailyCostsWithoutTags.Add(newItem);
}
return ExportToCsv(settings.SkipHeader, dailyCostsWithoutTags);
}
else
{
return ExportToCsv(settings.SkipHeader, dailyCosts);
}
}

public override Task WriteAnomalyDetectionResults(DetectAnomalySettings settings, List<AnomalyDetectionResult> anomalies)
Expand Down Expand Up @@ -141,7 +162,7 @@

public class CustomDoubleConverter : DoubleConverter
{
public override string ConvertToString(object value, IWriterRow row, MemberMapData memberMapData)

Check warning on line 165 in src/OutputFormatters/CsvOutputFormatter.cs

View workflow job for this annotation

GitHub Actions / build

Nullability of type of parameter 'value' doesn't match overridden member (possibly because of nullability attributes).
{
double number = (double)value;
return number.ToString("F8", CultureInfo.InvariantCulture);
Expand Down
29 changes: 21 additions & 8 deletions src/OutputFormatters/JsonOutputFormatter.cs
Original file line number Diff line number Diff line change
Expand Up @@ -65,16 +65,29 @@ public override Task WriteBudgets(BudgetsSettings settings, IEnumerable<BudgetIt
public override Task WriteDailyCost(DailyCostSettings settings, IEnumerable<CostDailyItem> dailyCosts)
{
// Create a new variable to hold the dailyCost items per day
var output = dailyCosts
// Code to avoid creating the column Tags when is not needed
if (settings.IncludeTags == false)
{
var output = dailyCosts
.GroupBy(a => a.Date)
.Select(a => new
{
Date = a.Key,
Items = a.Select(b => new { b.Name, b.Cost, b.Currency, b.CostUsd})
});
WriteJson(settings, output);
}
else
{
var output = dailyCosts
.GroupBy(a => a.Date)
.Select(a => new
{
Date = a.Key,
Items = a.Select(b => new { b.Name, b.Cost, b.Currency, b.CostUsd })
});

WriteJson(settings, output);

{
Date = a.Key,
Items = a.Select(b => new { b.Name, b.Cost, b.Currency, b.CostUsd, b.Tags})
});
WriteJson(settings, output);
}
return Task.CompletedTask;
}

Expand Down
2 changes: 1 addition & 1 deletion src/OutputFormatters/MarkdownOutputFormatter.cs
Original file line number Diff line number Diff line change
Expand Up @@ -301,7 +301,7 @@ public override Task WriteDailyCost(DailyCostSettings settings, IEnumerable<Cost
var othersCost = day.Except(topCosts)
.Sum(item => settings.UseUSD ? item.CostUsd : item.Cost);

topCosts.Add(new CostDailyItem(day.Key, "Other", othersCost, othersCost, day.First().Currency));
topCosts.Add(new CostDailyItem(day.Key, "Other", othersCost, othersCost, day.First().Currency, ""));

var dailyCost = 0D; // Keep track of the total cost for this day
var breakdown = new List<string>();
Expand Down
2 changes: 1 addition & 1 deletion src/OutputFormatters/TextOutputFormatter.cs
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
var currency =settings.UseUSD ? "USD" : accumulatedCostDetails.Costs.FirstOrDefault()?.Currency;

Console.WriteLine(
$"Azure Cost Overview for {accumulatedCostDetails.Subscription.displayName} from {accumulatedCostDetails.Costs.Min(a => a.Date)} to {accumulatedCostDetails.Costs.Max(a => a.Date)}");

Check warning on line 36 in src/OutputFormatters/TextOutputFormatter.cs

View workflow job for this annotation

GitHub Actions / build

Dereference of a possibly null reference.
Console.WriteLine();
Console.WriteLine("Totals:");
Console.WriteLine($" Today: {output.costs.todaysCost:N2} {currency}");
Expand Down Expand Up @@ -178,7 +178,7 @@
var othersCost = day.Except(topCosts)
.Sum(item => settings.UseUSD ? item.CostUsd : item.Cost);

topCosts.Add(new CostDailyItem(day.Key, "Other", othersCost, othersCost, day.First().Currency));
topCosts.Add(new CostDailyItem(day.Key, "Other", othersCost, othersCost, day.First().Currency, ""));

Console.Write($"{day.Key.ToString(CultureInfo.CurrentCulture)} ");

Expand Down
Loading