Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PC-10133: Move SLO into separate package #141

Merged
merged 105 commits into from
Nov 20, 2023
Merged
Show file tree
Hide file tree
Changes from 98 commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
dff641f
add validation poc
nieomylnieja Sep 28, 2023
e63773a
further refine the validation example
nieomylnieja Sep 28, 2023
3008897
current progress
nieomylnieja Sep 28, 2023
cdeaa57
simplify rule logic
nieomylnieja Sep 28, 2023
f4ed214
remove v1alpha aliases
nieomylnieja Sep 28, 2023
73e3118
add errors tests
nieomylnieja Sep 29, 2023
9ea3356
add rules tests
nieomylnieja Sep 29, 2023
4aee34f
add strings tests
nieomylnieja Sep 29, 2023
cbc92d7
make sure labels testing is deterministic
nieomylnieja Sep 29, 2023
520dbe9
invert dependency between project and v1alpha
nieomylnieja Oct 2, 2023
219a643
facilitate recent suggestions and feedback
nieomylnieja Oct 2, 2023
a4e8ff8
rename object validation to struct
nieomylnieja Oct 2, 2023
1898c1c
fix error handling
nieomylnieja Oct 3, 2023
d297d31
change labels validation rule name
nieomylnieja Oct 3, 2023
19426e3
let compiler infer the type
nieomylnieja Oct 4, 2023
83e31b7
declare the function once
nieomylnieja Oct 4, 2023
54de7fe
move service to separate pkg
nieomylnieja Oct 3, 2023
deacb28
extract common validation
nieomylnieja Oct 3, 2023
c13cde9
use common validators
nieomylnieja Oct 4, 2023
cbb67f8
revert changes
nieomylnieja Oct 4, 2023
f653b5a
move replya away
nieomylnieja Oct 4, 2023
f75914e
come up with extended API for validaiton
nieomylnieja Oct 9, 2023
c67d060
remove unneccessary level
nieomylnieja Oct 9, 2023
82abba1
refactor naming to match fluent
nieomylnieja Oct 9, 2023
644d3ba
move replay
nieomylnieja Oct 10, 2023
80bba4f
extend tests coverage
nieomylnieja Oct 10, 2023
e5bc3f8
move sli analysis away for time being
nieomylnieja Oct 10, 2023
57dad39
fix replya status through redefinition
nieomylnieja Oct 10, 2023
c33cb88
fix prepending name solution
nieomylnieja Oct 11, 2023
ba14f49
current progress
nieomylnieja Oct 3, 2023
d4b672d
split validator
nieomylnieja Oct 11, 2023
710176c
basic metadata validation
nieomylnieja Oct 11, 2023
eda43d1
correct expected metadata test result
nieomylnieja Oct 11, 2023
f56a94d
correct example
nieomylnieja Oct 11, 2023
8716035
add budgeting method validation
nieomylnieja Oct 11, 2023
e53cd56
post merge fixes
nieomylnieja Oct 11, 2023
7fcd3d6
introduce foreach validation
nieomylnieja Oct 12, 2023
8a4e61f
test spec.service and spec.alertPolicies
nieomylnieja Oct 12, 2023
e852925
adjust for each errors
nieomylnieja Oct 13, 2023
6009e55
add url validation
nieomylnieja Oct 13, 2023
77abdb7
validate attachments
nieomylnieja Oct 13, 2023
4cf4e6c
add oneof validation
nieomylnieja Oct 13, 2023
e6bf7fb
validate composite
nieomylnieja Oct 13, 2023
6b453b6
add extended length rules
nieomylnieja Oct 13, 2023
747041b
add unique rule and finish up anomaly config testing
nieomylnieja Oct 13, 2023
bf5d6df
add time window validation
nieomylnieja Oct 17, 2023
7d7c727
add basic indicator validation
nieomylnieja Oct 17, 2023
6e36de2
add new instructions around required and omitempty
nieomylnieja Oct 18, 2023
9595099
current progress
nieomylnieja Oct 21, 2023
a6de76e
refactor errors handling
nieomylnieja Oct 25, 2023
85e8198
correct tests
nieomylnieja Oct 25, 2023
da2f4f0
fix linter
nieomylnieja Oct 25, 2023
ae230a3
finish up with general metric spec validation
nieomylnieja Oct 26, 2023
59f522b
kinda finish appdynamics
nieomylnieja Oct 26, 2023
2ad23b4
corrrect appd validation
nieomylnieja Oct 27, 2023
0c9fb32
current lighstep progress
nieomylnieja Oct 28, 2023
6f9ff98
lighstep finish up
nieomylnieja Oct 29, 2023
71cca76
finish up lighstep validation
nieomylnieja Oct 30, 2023
20beb46
init pingdom work
nieomylnieja Oct 30, 2023
cae6ef1
add pingdom validation
nieomylnieja Oct 30, 2023
4e26a70
add sumologic rules
nieomylnieja Oct 30, 2023
73c83e6
add sumologic tests
nieomylnieja Oct 31, 2023
4f2ba11
finish sumologic testing
nieomylnieja Oct 31, 2023
7404406
add thousand eyes validation
nieomylnieja Oct 31, 2023
b05c196
finish AzureMonitor validation
nieomylnieja Oct 31, 2023
43f7b5c
bring back bad/good validation
nieomylnieja Oct 31, 2023
c8bf093
finish instana validation
nieomylnieja Nov 2, 2023
3c63eb7
add redshift validation
nieomylnieja Nov 2, 2023
7e26a3c
add bigquery validation
nieomylnieja Nov 2, 2023
b735830
add cloudwatch validation
nieomylnieja Nov 3, 2023
5f5024d
extract the rest of metrics
nieomylnieja Nov 3, 2023
b9aa40f
add prometheuses and datadog
nieomylnieja Nov 4, 2023
779ea31
add dynatrace, elastic, gcm, graphite and influx
nieomylnieja Nov 4, 2023
3990309
add loki, newrelic, opentsdb, splunkobs
nieomylnieja Nov 4, 2023
e5c5e3a
Merge remote-tracking branch 'origin/main' into PC-10133-move-slo-int…
nieomylnieja Nov 4, 2023
96eb10d
post merge addons
nieomylnieja Nov 6, 2023
7a16a76
fix linter
nieomylnieja Nov 6, 2023
ccf94a9
remove old validator from slo pkg
nieomylnieja Nov 6, 2023
b1ca81a
rename big query file
nieomylnieja Nov 6, 2023
1098876
brin back aws account id
nieomylnieja Nov 6, 2023
8369f6b
move time window
nieomylnieja Nov 6, 2023
3888ba2
add helper functions for value and target
nieomylnieja Nov 6, 2023
e767807
add tests recording capabilities
nieomylnieja Nov 6, 2023
1476cde
add ability to record tests
nieomylnieja Nov 6, 2023
b040d4b
use octalliteral
nieomylnieja Nov 6, 2023
d0ab019
fix gosec
nieomylnieja Nov 7, 2023
d4f1663
improve recorded code
nieomylnieja Nov 7, 2023
babd694
make accountId optional for cloudwatch
nieomylnieja Nov 8, 2023
2e5f914
cr suggestions
nieomylnieja Nov 9, 2023
10c32bf
apply cr suggestions
nieomylnieja Nov 13, 2023
844fedf
Merge remote-tracking branch 'origin/main' into PC-10133-move-slo-int…
nieomylnieja Nov 13, 2023
031d9cb
fix pingdom required validation and record golden paths
nieomylnieja Nov 14, 2023
0d25bf5
add record target
nieomylnieja Nov 14, 2023
e20babb
Merge remote-tracking branch 'origin/main' into PC-10133-move-slo-int…
nieomylnieja Nov 14, 2023
96a6f80
post merge fix
nieomylnieja Nov 14, 2023
de83a1b
cr suggestions
nieomylnieja Nov 14, 2023
df18924
fix newline handling for values
nieomylnieja Nov 14, 2023
e6ba19c
Merge remote-tracking branch 'origin/main' into PC-10133-move-slo-int…
nieomylnieja Nov 14, 2023
10f6f23
Merge remote-tracking branch 'origin/main' into PC-10133-move-slo-int…
nieomylnieja Nov 16, 2023
45805f1
fix expresion typo
nieomylnieja Nov 16, 2023
11f1eb6
c8a7ec3e9c52b06df507c4268f7af78434f714a1 commit changes
nieomylnieja Nov 16, 2023
8f679e1
migrate honeycomb and improve datasources handling
nieomylnieja Nov 17, 2023
5ab20b7
fix typo
nieomylnieja Nov 17, 2023
954093e
Merge remote-tracking branch 'origin/main' into PC-10133-move-slo-int…
nieomylnieja Nov 20, 2023
8be8538
fix yarn version
nieomylnieja Nov 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ linters-settings:
default-signifies-exhaustive: true
misspell:
locale: US
gocognit:
min-complexity: 30

linters:
disable-all: true
Expand Down
8 changes: 7 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,17 @@ define _print_check_step
printf -- '------\n%s...\n' "${1}"
endef

.PHONY: test
.PHONY: test test/record
## Run all unit tests.
test:
go test -race -cover ./...

## Record tests and save them in ./bin/recorded-tests.json.
test/record:
RECORD_FILE="$(abspath $(dir .))/$(BIN_DIR)/recorded-tests" ; \
NOBL9_SDK_TEST_RECORD_FILE="$$RECORD_FILE" go test ./... ; \
jq -s < "$$RECORD_FILE" > "$$RECORD_FILE.json"

.PHONY: check check/vet check/lint check/gosec check/spell check/trailing check/markdown check/renovate check/format check/generate check/vulns
## Run all checks.
check: check/vet check/lint check/gosec check/spell check/trailing check/markdown check/renovate check/format check/generate check/vulns
Expand Down
4 changes: 4 additions & 0 deletions cspell.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,10 @@ ignorePaths:
- bin/**
# TODO: Remove this exclusion after cleanup is done.
- manifest/**
- manifest/v1alpha/slo/validation_test.go
- validation/string_test.go
words:
- Equalf
- GOFILE
- GOPACKAGE
- Structs
Expand Down Expand Up @@ -57,6 +60,7 @@ words:
- promql
- puml
- rollup
- rtmp
- slis
- sloctl
- sloerrorbudgetstatus
Expand Down
101 changes: 101 additions & 0 deletions internal/testutils/assert.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
package testutils

import (
"encoding/json"
"strings"
"testing"

"github.com/stretchr/testify/require"

"github.com/nobl9/nobl9-go/manifest/v1alpha"
"github.com/nobl9/nobl9-go/validation"
)

type ExpectedError struct {
Prop string `json:"property"`
Code string `json:"code,omitempty"`
Message string `json:"message,omitempty"`
ContainsMessage string `json:"containsMessage,omitempty"`
}

// AssertNoError asserts that the provided v1alpha.ObjectError is nil.
func AssertNoError(t *testing.T, object interface{}, objErr *v1alpha.ObjectError) {
t.Helper()
rec.Record(t, object, 0, nil)

require.Nil(t, objErr, "ObjectError is expected to be nil")
}

// AssertContainsErrors asserts that the given object has:
// - the expected number of errors
// - at least one error which matches ExpectedError
//
// ExpectedError and actual error are considered equal if they point at the same property and either:
// - validation.ErrorCode are equal
// - error messages re equal
// - ExpectedError.ContainsMessage is contained in actual error message
//
// nolint: gocognit
func AssertContainsErrors(
t *testing.T,
object interface{},
objErr *v1alpha.ObjectError,
expectedErrorsCount int,
expectedErrors ...ExpectedError,
) {
t.Helper()
rec.Record(t, object, expectedErrorsCount, expectedErrors)

require.NotNil(t, objErr, "ObjectError is expected but got nil")
// Count errors.
actualErrorsCount := 0
for _, actual := range objErr.Errors {
var propErr *validation.PropertyError
require.ErrorAs(t, actual, &propErr)
actualErrorsCount += len(propErr.Errors)
}
require.Equalf(t,
expectedErrorsCount,
actualErrorsCount,
"%T contains a different number of errors than expected", objErr)
// Find and match expected errors.
for _, expected := range expectedErrors {
found := false
searchErrors:
for _, actual := range objErr.Errors {
var propErr *validation.PropertyError
require.ErrorAs(t, actual, &propErr)
if propErr.PropertyName != expected.Prop {
continue
}
for _, actualRuleErr := range propErr.Errors {
if expected.Message != "" && expected.Message == actualRuleErr.Message {
found = true
break searchErrors
}
if expected.ContainsMessage != "" && strings.Contains(actualRuleErr.Message, expected.ContainsMessage) {
found = true
break searchErrors
}
if expected.Code != "" &&
(expected.Code == actualRuleErr.Code || validation.HasErrorCode(actualRuleErr, expected.Code)) {
found = true
break searchErrors
}
}
}
// Pretty print the diff.
encExpected, _ := json.MarshalIndent(expected, "", " ")
encActual, _ := json.MarshalIndent(objErr.Errors, "", " ")
require.Truef(t, found,
"expected error was not found\nEXPECTED:\n%s\nENCOUNTERED:\n%s",
string(encExpected), string(encActual))
}
}

func PrependPropertyPath(errs []ExpectedError, path string) []ExpectedError {
for i := range errs {
errs[i].Prop = path + "." + errs[i].Prop
}
return errs
}
7 changes: 7 additions & 0 deletions internal/testutils/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
// Package testutils provides utility functions for testing.
// It also allows recording tests, both their input and expected output.
// The recorded test is then saved to a JSON file which is provided by
// the NOBL9_SDK_TEST_RECORD_FILE environment variable.
// The env variable should be an absolute path, otherwise every package
// will have a separate file created.
package testutils
66 changes: 66 additions & 0 deletions internal/testutils/record.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
package testutils

import (
"encoding/json"
"io"
"os"
"sync"
"testing"

"github.com/rs/zerolog/log"
)

var rec = new(testRecorder)

type testRecorder struct {
shouldRecord bool
output io.Writer
mu sync.Mutex
initOnce sync.Once
}

type recordedTest struct {
TestName string `json:"testName"`
Object interface{} `json:"object"`
IsValid bool `json:"isValid"`
ErrorsCount int `json:"errorsCount,omitempty"`
Errors []ExpectedError `json:"errors,omitempty"`
}

func (r *testRecorder) Record(t *testing.T, object interface{}, errorsCount int, errors []ExpectedError) {
r.Init()
if !r.shouldRecord {
return
}
r.mu.Lock()
defer r.mu.Unlock()
rt := recordedTest{
TestName: t.Name(),
Object: object,
ErrorsCount: errorsCount,
Errors: errors,
}
if errorsCount == 0 {
rt.IsValid = true
}
if err := json.NewEncoder(r.output).Encode(rt); err != nil {
log.Err(err).Msg("failed to record test")
}
}

func (r *testRecorder) Init() {
r.initOnce.Do(func() {
path, isSet := os.LookupEnv("NOBL9_SDK_TEST_RECORD_FILE")
if !isSet {
return
}
r.shouldRecord = true
// #nosec G304
f, err := os.Create(path)
if err != nil {
panic(err)
}
r.output = f
log.Info().Msg("test recorder initialized, all test will be recorded in " + path)
})
}
6 changes: 3 additions & 3 deletions manifest/object_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,20 +46,20 @@ var expectedUniquenessConstraintMessage string
func TestValidate(t *testing.T) {
t.Run("nil objects slice", func(t *testing.T) {
errs := Validate(nil)
assert.Empty(t, errs)
assert.Nil(t, errs)
})

t.Run("empty objects slice", func(t *testing.T) {
errs := Validate([]Object{})
assert.Empty(t, errs)
assert.Nil(t, errs)
})

t.Run("no errors", func(t *testing.T) {
errs := Validate([]Object{
customObject{kind: KindProject, name: "default"},
customObject{kind: KindRoleBinding, name: "default"},
})
assert.Empty(t, errs)
assert.Nil(t, errs)
})

t.Run("errors", func(t *testing.T) {
Expand Down
6 changes: 0 additions & 6 deletions manifest/v1alpha/agent.go
Original file line number Diff line number Diff line change
Expand Up @@ -251,9 +251,3 @@ type GenericAgentConfig struct {
type HoneycombAgentConfig struct {
// Honeycomb agent doesn't require any additional parameters.
}

// AgentWithSLOs struct which mapped one to one with kind: agent and slo yaml definition
type AgentWithSLOs struct {
Agent Agent `json:"agent"`
SLOs []SLO `json:"slos"`
}
6 changes: 0 additions & 6 deletions manifest/v1alpha/alert_policy.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,3 @@ type AlertCondition struct {
LastsForDuration string `json:"lastsFor,omitempty" validate:"omitempty,validDuration,nonNegativeDuration" example:"15m"` //nolint:lll
Operator string `json:"op,omitempty" validate:"omitempty,operator" example:"lt"`
}

// AlertPolicyWithSLOs struct which mapped one to one with kind: alert policy and slo yaml definition
type AlertPolicyWithSLOs struct {
AlertPolicy AlertPolicy `json:"alertPolicy"`
SLOs []SLO `json:"slos"`
}
2 changes: 1 addition & 1 deletion manifest/v1alpha/aws.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ func AWSRegions() []AWSRegion {
}
}

func isValidRegion(code string, regions []AWSRegion) bool {
func IsValidRegion(code string, regions []AWSRegion) bool {
for _, region := range regions {
if region.Code == code {
return true
Expand Down
6 changes: 0 additions & 6 deletions manifest/v1alpha/direct.go
Original file line number Diff line number Diff line change
Expand Up @@ -424,9 +424,3 @@ type HoneycombDirectConfig struct {
type PublicHoneycombDirectConfig struct {
HiddenAPIKey string `json:"apiKey,omitempty"`
}

// PublicDirectWithSLOs struct which mapped one to one with kind: direct and slo yaml definition
type PublicDirectWithSLOs struct {
Direct PublicDirect `json:"direct"`
SLOs []SLO `json:"slos"`
}
Loading
Loading