Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/dashboard command #10

Merged
merged 9 commits into from
Nov 14, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
123 changes: 109 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,25 +31,53 @@ $ brew install mashiike/tap/shimesaba
### as CLI command

```console
$ shimesaba -config config.yaml -mackerel-apikey <Mackerel API Key>
$ shimesaba -config config.yaml -mackerel-apikey <Mackerel API Key> run
```

```console
NAME:
shimesaba - A commandline tool for tracking SLO/ErrorBudget using Mackerel as an SLI measurement service.

USAGE:
shimesaba [global options] command [command options] [arguments...]

VERSION:
current

COMMANDS:
dashboard manage mackerel dashboard for SLI/SLO
run run shimesaba. this is main feature
help, h Shows a list of commands or help for one command

GLOBAL OPTIONS:
--config value, -c value config file path, can set multiple [$CONFIG, $SHIMESABA_CONFIG]
--debug output debug log (default: false) [$SHIMESABA_DEBUG]
--mackerel-apikey value, -k value for access mackerel API (default: *********) [$MACKEREL_APIKEY, $SHIMESABA_MACKEREL_APIKEY]
--help, -h show help (default: false)
--version, -v print the version (default: false)
2021/11/14 23:29:45 [error] Required flag "config" not set
```
Usage of shimesaba:
-backfill uint
generate report before n point (default 3)
-config value
config file path, can set multiple
-debug
output debug log
-dry-run
report output stdout and not put mackerel
-mackerel-apikey string
for access mackerel API

run command usage is follow
```console
$ shimesaba run --help
NAME:
main run - run shimesaba. this is main feature

USAGE:
shimesaba -config <config file> run [command options]

OPTIONS:
--dry-run report output stdout and not put mackerel (default: false) [$SHIMESABA_DRY_RUN]
--backfill value generate report before n point (default: 3) [$BACKFILL, $SHIMESABA_BACKFILL]
--help, -h show help (default: false)
```

### as AWS Lambda function

`shimesaba` binary also runs as AWS Lambda function.
`shimesaba` binary also runs as AWS Lambda function.
shimesaba implicitly behaves as a run command when run as a bootstrap with a Lambda Function


CLI options can be specified from environment variables. For example, when `MACKEREL_APIKEY` environment variable is set, the value is set to `-mackerel-apikey` option.

Expand All @@ -60,7 +88,7 @@ Example Lambda functions configuration.
"FunctionName": "shimesaba",
"Environment": {
"Variables": {
"CONFIG": "config.yaml",
"SHIMESABA_CONFIG": "config.yaml",
"MACKEREL_APIKEY": "<Mackerel API KEY>"
}
},
Expand Down Expand Up @@ -103,6 +131,8 @@ definitions:
objectives:
- expr: alb_p90_response_time <= 1.0
- expr: component_response_time <= 1.0

dashboard: dashboard.jsonnet
```

#### required_version
Expand Down Expand Up @@ -236,6 +266,71 @@ It incorporates [github.com/handlename/ssmwrap](https://github.com/handlename/ss
If you specify the path of the Parameter Store of AWS Systems Manager separated by commas, it will be output to the environment variable.
Useful when used as a Lambda function.

### Usage Dashboard subcommand.

This subcommand can only be used when acting as a CLI.
If the dashboard of the config file contains the dashboard definition file, you can manage the dashboard JSON using Go Template.

For example, you can build a simple dashboard by defining a json file like the one below.

dashboard.jsonnet
```jsonnet
local errorBudgetCounter(x, y, def_id, title) = {
type: 'value',
title: title,
layout: {
x: x,
y: y,
width: 10,
height: 5,
},
metric: {
type: 'service',
name: 'shimesaba.error_budget.' + def_id,
serviceName: 'shimesaba',
},
graph: null,
range: null,
fractionSize: 0,
suffix: 'min',
};
{
title: 'SLI/SLO',
urlPath: '4oequPJEwwd',
memo: '',
widgets: [
errorBudgetCounter(0, 0, 'availability', ''),
errorBudgetCounter(10, 0, 'latency', ''),
{
type: 'markdown',
title: 'SLO Definitions',
layout: {
x: 20,
y: 0,
width: 5,
height: 20,
},
markdown: '{{file `definitions.md` | json_escape }}',
},
],
}
```

definitions.md
```markdown
{{ range $def_id, $def := .Definitions }}
## SLO {{ $def_id }}

- TimeFrame : {{ $def.TimeFrame }}
- ErrorBudgetSize: {{ $def.ErrorBudgetSizeDuration }}


{{ range $def.Objectives }}
- {{ . }}
{{ end }}
{{ end }}
```

## LICENSE

MIT
57 changes: 16 additions & 41 deletions app.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
"errors"
"fmt"
"log"
"path/filepath"
"sort"
"time"

Expand All @@ -20,8 +21,10 @@ type App struct {
metricConfigs MetricConfigs
definitions []*Definition

maxTimeFrame time.Duration
maxCalculate time.Duration
maxTimeFrame time.Duration
maxCalculate time.Duration
cfgPath string
dashboardPath string
}

//New creates an app
Expand Down Expand Up @@ -55,57 +58,29 @@ func NewWithMackerelClient(client MackerelClient, cfg *Config) (*App, error) {
definitions: definitions,
maxTimeFrame: maxTimeFrame,
maxCalculate: maxCalculate,
cfgPath: cfg.configFilePath,
dashboardPath: filepath.Join(cfg.configFilePath, cfg.Dashboard),
}
return app, nil
}

type runConfig struct {
dryRun bool
backfill int
}

//RunOption is an App.Run option
type RunOption interface {
apply(*runConfig)
}

type runOptionFunc func(*runConfig)

func (f runOptionFunc) apply(rc *runConfig) {
f(rc)
}

//DryRunOption is an option to output the calculated error budget as standard without posting it to Mackerel.
func DryRunOption(dryRun bool) RunOption {
return runOptionFunc(func(rc *runConfig) {
rc.dryRun = dryRun
})
}

//BackfillOption specifies how many points of data to calculate retroactively from the current time.
func BackfillOption(count int) RunOption {
return runOptionFunc(func(rc *runConfig) {
rc.backfill = count
})
}

//Run performs the calculation of the error bar calculation
func (app *App) Run(ctx context.Context, opts ...RunOption) error {
func (app *App) Run(ctx context.Context, optFns ...func(*Options)) error {
log.Printf("[info] start run")
rc := &runConfig{
opts := &Options{
backfill: 3,
dryRun: false,
}
for _, opt := range opts {
opt.apply(rc)
for _, optFn := range optFns {
optFn(opts)
}
if rc.backfill <= 0 {
if opts.backfill <= 0 {
return errors.New("backfill must over 0")
}
log.Println("[debug]", app.metricConfigs)
now := flextime.Now()
startAt := now.Truncate(app.maxCalculate).
Add(-(time.Duration(rc.backfill))*app.maxCalculate - app.maxTimeFrame).
Add(-(time.Duration(opts.backfill))*app.maxCalculate - app.maxTimeFrame).
Truncate(app.maxCalculate)
log.Printf("[info] fetch metric range %s ~ %s", startAt, now)
metrics, err := app.repo.FetchMetrics(ctx, app.metricConfigs, startAt, now)
Expand All @@ -119,17 +94,17 @@ func (app *App) Run(ctx context.Context, opts ...RunOption) error {
if err != nil {
return fmt.Errorf("objective[%s] create report failed: %w", d.ID(), err)
}
if len(reports) > rc.backfill {
if len(reports) > opts.backfill {
sort.Slice(reports, func(i, j int) bool {
return reports[i].DataPoint.Before(reports[j].DataPoint)
})
n := len(reports) - rc.backfill
n := len(reports) - opts.backfill
if n < 0 {
n = 0
}
reports = reports[n:]
}
if rc.dryRun {
if opts.dryRun {
log.Printf("[info] dryrun! output stdout reports[%s]\n", d.ID())
bs, err := json.MarshalIndent(reports, "", " ")
if err != nil {
Expand Down
Loading