Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rate aggregation #61369

Merged
merged 6 commits into from
Aug 25, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/reference/aggregations/metrics.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ include::metrics/valuecount-aggregation.asciidoc[]

include::metrics/t-test-aggregation.asciidoc[]

include::metrics/rate-aggregation.asciidoc[]



Expand Down
257 changes: 257 additions & 0 deletions docs/reference/aggregations/metrics/rate-aggregation.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,257 @@
[role="xpack"]
[testenv="basic"]
[[search-aggregations-metrics-rate-aggregation]]
=== Rate Aggregation

A `rate` metrics aggregation can be used only inside a `date_histogram` and calculates a rate of documents or a field in each
`date_histogram` bucket.

==== Syntax

A `rate` aggregation looks like this in isolation:

[source,js]
--------------------------------------------------
{
"rate": {
"unit": "month",
"field": "requests"
}
}
--------------------------------------------------
// NOTCONSOLE

The following request will group all sales records into monthly bucket and than convert the number of sales transaction in each bucket
into per annual sales rate.

[source,console]
--------------------------------------------------
GET sales/_search
{
"size": 0,
"aggs": {
"by_date": {
"date_histogram": {
"field": "date",
"calendar_interval": "month" <1>
},
"aggs": {
"my_rate": {
"rate": {
"unit": "year" <2>
}
}
}
}
}
}
--------------------------------------------------
// TEST[setup:sales]
<1> Histogram is grouped by month.
<2> But the rate is converted into annual rate.

The response will return the annual rate of transaction in each bucket. Since there are 12 months per year, the annual rate will
be automatically calculated by multiplying monthly rate by 12.

[source,console-result]
--------------------------------------------------
{
...
"aggregations" : {
"by_date" : {
"buckets" : [
{
"key_as_string" : "2015/01/01 00:00:00",
"key" : 1420070400000,
"doc_count" : 3,
"my_rate" : {
"value" : 36.0
}
},
{
"key_as_string" : "2015/02/01 00:00:00",
"key" : 1422748800000,
"doc_count" : 2,
"my_rate" : {
"value" : 24.0
}
},
{
"key_as_string" : "2015/03/01 00:00:00",
"key" : 1425168000000,
"doc_count" : 2,
"my_rate" : {
"value" : 24.0
}
}
]
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\./"took": $body.took,"timed_out": false,"_shards": $body._shards,"hits": $body.hits,/]

Instead of counting the number of documents, it is also possible to calculate a sum of all values of the fields in the documents in each
bucket. The following request will group all sales records into monthly bucket and than calculate the total monthly sales and convert them
into average daily sales.

[source,console]
--------------------------------------------------
GET sales/_search
{
"size": 0,
"aggs": {
"by_date": {
"date_histogram": {
"field": "date",
"calendar_interval": "month" <1>
},
"aggs": {
"avg_price": {
"rate": {
"field": "price", <2>
"unit": "day" <3>
}
}
}
}
}
}
--------------------------------------------------
// TEST[setup:sales]
<1> Histogram is grouped by month.
<2> Calculate sum of all sale prices
<3> Convert to average daily sales

The response will contain the average daily sale prices for each month.

[source,console-result]
--------------------------------------------------
{
...
"aggregations" : {
"by_date" : {
"buckets" : [
{
"key_as_string" : "2015/01/01 00:00:00",
"key" : 1420070400000,
"doc_count" : 3,
"avg_price" : {
"value" : 17.741935483870968
}
},
{
"key_as_string" : "2015/02/01 00:00:00",
"key" : 1422748800000,
"doc_count" : 2,
"avg_price" : {
"value" : 2.142857142857143
}
},
{
"key_as_string" : "2015/03/01 00:00:00",
"key" : 1425168000000,
"doc_count" : 2,
"avg_price" : {
"value" : 12.096774193548388
}
}
]
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\./"took": $body.took,"timed_out": false,"_shards": $body._shards,"hits": $body.hits,/]


==== Relationship between bucket sizes and rate

The `rate` aggregation supports all rate that can be used <<calendar_intervals,calendar_intervals parameter>> of `date_histogram`
aggregation. The specified rate should compatible with the `date_histogram` aggregation interval, i.e. it should be possible to
convert the bucket size into the rate. By default the interval of the `date_histogram` is used.

`"rate": "second"`:: compatible with all intervals
`"rate": "minute"`:: compatible with all intervals
`"rate": "hour"`:: compatible with all intervals
`"rate": "day"`:: compatible with all intervals
`"rate": "week"`:: compatible with all intervals
`"rate": "month"`:: compatible with only with `month`, `quarter` and `year` calendar intervals
`"rate": "quarter"`:: compatible with only with `month`, `quarter` and `year` calendar intervals
`"rate": "year"`:: compatible with only with `month`, `quarter` and `year` calendar intervals

==== Script

The `rate` aggregation also supports scripting. For example, if we need to adjust out prices before calculating rates, we could use
a script to recalculate them on-the-fly:

[source,console]
--------------------------------------------------
GET sales/_search
{
"size": 0,
"aggs": {
"by_date": {
"date_histogram": {
"field": "date",
"calendar_interval": "month"
},
"aggs": {
"avg_price": {
"rate": {
"script": { <1>
"lang": "painless",
"source": "doc['price'].value * params.adjustment",
"params": {
"adjustment": 0.9 <2>
}
}
}
}
}
}
}
}
--------------------------------------------------
// TEST[setup:sales]

<1> The `field` parameter is replaced with a `script` parameter, which uses the
script to generate values which percentiles are calculated on.
<2> Scripting supports parameterized input just like any other script.

[source,console-result]
--------------------------------------------------
{
...
"aggregations" : {
"by_date" : {
"buckets" : [
{
"key_as_string" : "2015/01/01 00:00:00",
"key" : 1420070400000,
"doc_count" : 3,
"avg_price" : {
"value" : 495.0
}
},
{
"key_as_string" : "2015/02/01 00:00:00",
"key" : 1422748800000,
"doc_count" : 2,
"avg_price" : {
"value" : 54.0
}
},
{
"key_as_string" : "2015/03/01 00:00:00",
"key" : 1425168000000,
"doc_count" : 2,
"avg_price" : {
"value" : 337.5
}
}
]
}
}
}
--------------------------------------------------
// TESTRESPONSE[s/\.\.\./"took": $body.took,"timed_out": false,"_shards": $body._shards,"hits": $body.hits,/]
1 change: 1 addition & 0 deletions docs/reference/rest-api/usage.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,7 @@ GET /_xpack/usage
"normalize_usage" : 0,
"cumulative_cardinality_usage" : 0,
"t_test_usage" : 0,
"rate_usage" : 0,
"string_stats_usage" : 0,
"moving_percentiles_usage" : 0
}
Expand Down
Loading