Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pkg/ottl] Add support for scaling values #33246

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
b782416
[processor/transform] Add support for scaling values
bacherfl May 27, 2024
adcda48
revert change to e2e test
bacherfl May 28, 2024
4ebb38a
rename helper function
bacherfl May 28, 2024
3ba07ac
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl May 28, 2024
d610bf0
add documentation for Scale function
bacherfl May 28, 2024
9d741ac
Merge remote-tracking branch 'bacherfl/feat/16214/scale-metric-values…
bacherfl May 28, 2024
d098a77
adapt to possible types returned by metric context path
bacherfl May 28, 2024
f019ec0
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl May 28, 2024
15a2006
additional test for exponential histogram
bacherfl May 28, 2024
213fcce
Merge remote-tracking branch 'bacherfl/feat/16214/scale-metric-values…
bacherfl May 28, 2024
285a37d
use type switch for handling different types as Scale argument
bacherfl May 29, 2024
fc1dc7f
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl May 29, 2024
4084b53
incorporate feedback from PR review
bacherfl Jun 3, 2024
35064e1
Merge remote-tracking branch 'bacherfl/feat/16214/scale-metric-values…
bacherfl Jun 3, 2024
343ce4e
add license comment
bacherfl Jun 3, 2024
549bdc3
address PR review comments, add changelog entry
bacherfl Jun 4, 2024
fb04e21
fix linting
bacherfl Jun 5, 2024
75e24b1
Merge branch 'main' into feat/16214/scale-metric-values
evan-bradley Jun 5, 2024
49321a2
Update pkg/ottl/ottlfuncs/func_scale.go
bacherfl Jun 6, 2024
47d6834
fix e2e tests
bacherfl Jun 6, 2024
313cd7e
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl Jun 6, 2024
d76e6cc
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl Jun 13, 2024
44997a1
Merge branch 'main' into feat/16214/scale-metric-values
evan-bradley Jun 14, 2024
0f07a49
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl Jun 17, 2024
92d33ed
adapt scale function to be an editor function
bacherfl Jun 17, 2024
f660ca6
remove unused testing struct
bacherfl Jun 17, 2024
37f16bb
remove unused testing struct; adapt test
bacherfl Jun 17, 2024
62a539b
adapt test
bacherfl Jun 17, 2024
61e2a09
fix linting
bacherfl Jun 17, 2024
75a6d67
adapt the function to only work with metrics
bacherfl Jun 18, 2024
14bbdd8
remove obsolete test
bacherfl Jun 18, 2024
0090fcc
apply suggestions from PR review
bacherfl Jun 19, 2024
d398d64
move scale function to transformprocessor
bacherfl Jun 19, 2024
e495e03
fix failing checks
bacherfl Jun 20, 2024
c83e5ba
fix linting
bacherfl Jun 20, 2024
66c5bd5
appease linter
bacherfl Jun 20, 2024
bf746e7
make unit an optional value
bacherfl Jun 24, 2024
ffed4f1
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl Jun 24, 2024
719daca
adapt unit tests to recent changes
bacherfl Jun 24, 2024
721c537
apply suggestion from code review
bacherfl Jul 2, 2024
a6c39a1
adapt to pr review
bacherfl Jul 2, 2024
876afc1
Apply suggestions from code review
bacherfl Jul 23, 2024
42d3140
incorporate review comments
bacherfl Jul 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .chloggen/add-scale-function.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: processor/transform

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Add `scale_metric` function that scales all data points in a metric.

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [16214]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: []
16 changes: 16 additions & 0 deletions processor/transformprocessor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,7 @@ In addition to OTTL functions, the processor defines its own functions to help w
- [convert_summary_count_val_to_sum](#convert_summary_count_val_to_sum)
- [convert_summary_sum_val_to_sum](#convert_summary_sum_val_to_sum)
- [copy_metric](#copy_metric)
- [scale_metric](#scale_metric)

### convert_sum_to_gauge

Expand Down Expand Up @@ -347,6 +348,21 @@ Examples:

- `copy_metric(desc="new desc") where description == "old desc"`

### scale_metric

`scale_metric(factor, Optional[unit])`
evan-bradley marked this conversation as resolved.
Show resolved Hide resolved

The `scale_metric` function multiplies the values in the data points in the metric by the float value `factor`.
If the optional string `unit` is provided, the metric's unit will be set to this value.
The supported data types are:

Supported metric types are `Gauge`, `Sum`, `Histogram`, and `Summary`.

Examples:

- `scale_metric(0.1)`: Scale the metric by a factor of `0.1`. The unit of the metric will not be modified.
- `scale_metric(10.0, "kWh")`: Scale the metric by a factor of `10.0` and sets the unit to `kWh`.

## Examples

### Perform transformation if field does not exist
Expand Down
130 changes: 130 additions & 0 deletions processor/transformprocessor/internal/metrics/func_scale.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package metrics // import "github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor/internal/metrics"

import (
"context"
"errors"
"fmt"

"go.opentelemetry.io/collector/pdata/pmetric"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl"
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/contexts/ottlmetric"
)

type ScaleArguments struct {
Multiplier float64
Unit ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]]
}

func newScaleMetricFactory() ottl.Factory[ottlmetric.TransformContext] {
return ottl.NewFactory("scale_metric", &ScaleArguments{}, createScaleFunction)
}

func createScaleFunction(_ ottl.FunctionContext, oArgs ottl.Arguments) (ottl.ExprFunc[ottlmetric.TransformContext], error) {
args, ok := oArgs.(*ScaleArguments)

if !ok {
return nil, fmt.Errorf("ScaleFactory args must be of type *ScaleArguments[K]")
}

return Scale(*args)
}

func Scale(args ScaleArguments) (ottl.ExprFunc[ottlmetric.TransformContext], error) {
return func(ctx context.Context, tCtx ottlmetric.TransformContext) (any, error) {
metric := tCtx.GetMetric()

var unit *string
if !args.Unit.IsEmpty() {
u, err := args.Unit.Get().Get(ctx, tCtx)
if err != nil {
return nil, fmt.Errorf("could not get unit from ScaleArguments: %w", err)
}
unit = &u
}

switch metric.Type() {
case pmetric.MetricTypeGauge:
scaleMetric(metric.Gauge().DataPoints(), args.Multiplier)
case pmetric.MetricTypeHistogram:
scaleHistogram(metric.Histogram().DataPoints(), args.Multiplier)
case pmetric.MetricTypeSummary:
scaleSummarySlice(metric.Summary().DataPoints(), args.Multiplier)
case pmetric.MetricTypeSum:
scaleMetric(metric.Sum().DataPoints(), args.Multiplier)
case pmetric.MetricTypeExponentialHistogram:
return nil, errors.New("exponential histograms are not supported by the 'scale_metric' function")
default:
return nil, fmt.Errorf("unsupported metric type: '%v'", metric.Type())
}
if unit != nil {
metric.SetUnit(*unit)
}

return nil, nil
}, nil
}

func scaleExemplar(ex *pmetric.Exemplar, multiplier float64) {
switch ex.ValueType() {
case pmetric.ExemplarValueTypeInt:
ex.SetIntValue(int64(float64(ex.IntValue()) * multiplier))
case pmetric.ExemplarValueTypeDouble:
ex.SetDoubleValue(ex.DoubleValue() * multiplier)
}
}

func scaleSummarySlice(values pmetric.SummaryDataPointSlice, multiplier float64) {
for i := 0; i < values.Len(); i++ {
dp := values.At(i)

dp.SetSum(dp.Sum() * multiplier)

for i := 0; i < dp.QuantileValues().Len(); i++ {
qv := dp.QuantileValues().At(i)
qv.SetValue(qv.Value() * multiplier)
}
}
}

func scaleHistogram(datapoints pmetric.HistogramDataPointSlice, multiplier float64) {
for i := 0; i < datapoints.Len(); i++ {
dp := datapoints.At(i)

if dp.HasSum() {
dp.SetSum(dp.Sum() * multiplier)
}
if dp.HasMin() {
dp.SetMin(dp.Min() * multiplier)
}
if dp.HasMax() {
dp.SetMax(dp.Max() * multiplier)
}

for bounds, bi := dp.ExplicitBounds(), 0; bi < bounds.Len(); bi++ {
bounds.SetAt(bi, bounds.At(bi)*multiplier)
}

for exemplars, ei := dp.Exemplars(), 0; ei < exemplars.Len(); ei++ {
exemplar := exemplars.At(ei)
scaleExemplar(&exemplar, multiplier)
}
}
}

func scaleMetric(points pmetric.NumberDataPointSlice, multiplier float64) {
for i := 0; i < points.Len(); i++ {
dp := points.At(i)
switch dp.ValueType() {
case pmetric.NumberDataPointValueTypeInt:
dp.SetIntValue(int64(float64(dp.IntValue()) * multiplier))

case pmetric.NumberDataPointValueTypeDouble:
dp.SetDoubleValue(dp.DoubleValue() * multiplier)
default:
}
}
}
204 changes: 204 additions & 0 deletions processor/transformprocessor/internal/metrics/func_scale_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,204 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package metrics

import (
"context"
"testing"

"github.com/stretchr/testify/assert"
"go.opentelemetry.io/collector/pdata/pcommon"
"go.opentelemetry.io/collector/pdata/pmetric"

"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl"
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/contexts/ottlmetric"
)

func TestScale(t *testing.T) {
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved
type testCase struct {
name string
args ScaleArguments
valueFunc func() pmetric.Metric
wantFunc func() pmetric.Metric
wantErr bool
}
tests := []testCase{
{
name: "scale gauge float metric",
valueFunc: func() pmetric.Metric {
metric := pmetric.NewMetric()
metric.SetName("test-metric")
metric.SetEmptyGauge()
metric.Gauge().DataPoints().AppendEmpty().SetDoubleValue(10.0)

return metric
},
args: ScaleArguments{
Multiplier: 10.0,
Unit: ottl.NewTestingOptional[ottl.StringGetter[ottlmetric.TransformContext]](ottl.StandardStringGetter[ottlmetric.TransformContext]{
Getter: func(_ context.Context, _ ottlmetric.TransformContext) (any, error) {
return "kWh", nil
},
}),
},
wantFunc: func() pmetric.Metric {
metric := pmetric.NewMetric()
metric.SetName("test-metric")
metric.SetEmptyGauge()
metric.SetUnit("kWh")
metric.Gauge().DataPoints().AppendEmpty().SetDoubleValue(100.0)

return metric
},
wantErr: false,
},
{
name: "scale gauge int metric",
valueFunc: func() pmetric.Metric {
metric := pmetric.NewMetric()
metric.SetName("test-metric")
metric.SetEmptyGauge()
metric.Gauge().DataPoints().AppendEmpty().SetIntValue(10)

return metric
},
args: ScaleArguments{
Multiplier: 10.0,
},
wantFunc: func() pmetric.Metric {
metric := pmetric.NewMetric()
metric.SetName("test-metric")
metric.SetEmptyGauge()
metric.Gauge().DataPoints().AppendEmpty().SetIntValue(100.0)

return metric
},
wantErr: false,
},
{
name: "scale sum metric",
valueFunc: func() pmetric.Metric {
metric := pmetric.NewMetric()
metric.SetName("test-metric")
metric.SetEmptySum()
metric.Sum().DataPoints().AppendEmpty().SetDoubleValue(10.0)

return metric
},
args: ScaleArguments{
Multiplier: 10.0,
},
wantFunc: func() pmetric.Metric {
metric := pmetric.NewMetric()
metric.SetName("test-metric")
metric.SetEmptySum()
metric.Sum().DataPoints().AppendEmpty().SetDoubleValue(100.0)

return metric
},
wantErr: false,
},
{
name: "scale histogram metric",
valueFunc: func() pmetric.Metric {
metric := getTestScalingHistogramMetric(1, 4, 1, 3, []float64{1, 10}, []uint64{1, 2}, []float64{1.0}, 1, 1)
return metric
},
args: ScaleArguments{
Multiplier: 10.0,
},
wantFunc: func() pmetric.Metric {
metric := getTestScalingHistogramMetric(1, 40, 10, 30, []float64{10, 100}, []uint64{1, 2}, []float64{10.0}, 1, 1)
return metric
},
wantErr: false,
},
{
name: "scale summary metric",
valueFunc: func() pmetric.Metric {
metric := pmetric.NewMetric()
dp := metric.SetEmptySummary().DataPoints().AppendEmpty()
dp.SetSum(10.0)
qv := dp.QuantileValues().AppendEmpty()
qv.SetValue(10.0)

return metric
},
args: ScaleArguments{
Multiplier: 10.0,
},
wantFunc: func() pmetric.Metric {
metric := pmetric.NewMetric()
dp := metric.SetEmptySummary().DataPoints().AppendEmpty()
dp.SetSum(100.0)
qv := dp.QuantileValues().AppendEmpty()
qv.SetValue(100.0)

return metric
},
wantErr: false,
},
{
name: "unsupported: exponential histogram metric",
valueFunc: func() pmetric.Metric {
metric := pmetric.NewMetric()
metric.SetEmptyExponentialHistogram()
return metric
},
args: ScaleArguments{
Multiplier: 10.0,
},
wantFunc: func() pmetric.Metric {
// value should not be modified
metric := pmetric.NewMetric()
metric.SetEmptyExponentialHistogram()
return metric
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
target := ottlmetric.NewTransformContext(
tt.valueFunc(),
pmetric.NewMetricSlice(),
pcommon.NewInstrumentationScope(),
pcommon.NewResource(),
pmetric.NewScopeMetrics(),
pmetric.NewResourceMetrics(),
)

expressionFunc, _ := Scale(tt.args)
_, err := expressionFunc(context.Background(), target)

if tt.wantErr {
assert.Error(t, err)
} else {
assert.NoError(t, err)
}
assert.EqualValues(t, tt.wantFunc(), target.GetMetric())
})
}
}

func getTestScalingHistogramMetric(count uint64, sum, min, max float64, bounds []float64, bucketCounts []uint64, exemplars []float64, start, timestamp pcommon.Timestamp) pmetric.Metric {
metric := pmetric.NewMetric()
metric.SetName("test-metric")
metric.SetEmptyHistogram()
histogramDatapoint := metric.Histogram().DataPoints().AppendEmpty()
histogramDatapoint.SetCount(count)
histogramDatapoint.SetSum(sum)
histogramDatapoint.SetMin(min)
histogramDatapoint.SetMax(max)
histogramDatapoint.ExplicitBounds().FromRaw(bounds)
histogramDatapoint.BucketCounts().FromRaw(bucketCounts)
for i := 0; i < len(exemplars); i++ {
exemplar := histogramDatapoint.Exemplars().AppendEmpty()
exemplar.SetTimestamp(1)
exemplar.SetDoubleValue(exemplars[i])
}
histogramDatapoint.SetStartTimestamp(start)
histogramDatapoint.SetTimestamp(timestamp)
return metric
}
Loading