Skip to content

Commit

Permalink
Added info about schema to the Metricbeat dev guide (#2057)
Browse files Browse the repository at this point in the history
  • Loading branch information
tsg authored and andrewkroh committed Jul 19, 2016
1 parent f016d36 commit 7bd61bf
Show file tree
Hide file tree
Showing 2 changed files with 69 additions and 2 deletions.
67 changes: 67 additions & 0 deletions metricbeat/docs/developer-guide/create-metricset.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,73 @@ For this kind of data, you can use the following `Fetch` interface:
The only difference between this and the previous example is that the second example returns `[]common.MapStr`.
Metricbeat will add the same timestamp to all the events in the list to make it possible to correlate the events.

[float]
==== Parsing and normalizing fields

In Metricbeat we aim to normalize the metric names from all Metricsets to
respect a common {libbeat}/event-conventions.html[set of conventions]. This
makes it easy for users to find and interpret metrics. To simplify parsing,
converting, renaming, and restructuring of the object read from the monitored
system to the Metricbeat format, we have created the
https://godoc.org/github.com/elastic/beats/metricbeat/schema[schema] package
that allows to declaratively define transformations.

For example, assuming this input object:

[source,go]
----
input := map[string]interface{}{
"testString": "hello",
"testInt": "42",
"testBool": "true",
"testFloat": "42.1",
"testObjString": "hello, object",
}
----

And the requirement to transform it into this one:

[source,go]
----
common.MapStr{
"test_string": "hello",
"test_int": int64(42),
"test_bool": true,
"test_float": 42.1,
"test_obj": common.MapStr{
"test_obj_string": "hello, object",
},
}
----

You can use the following code to make the transformations:

[source,go]
----
import (
s "github.com/elastic/beats/metricbeat/schema"
c "github.com/elastic/beats/metricbeat/schema/mapstrstr"
)
var (
schema = s.Schema{
"test_string": c.Str("testString"),
"test_int": c.Int("testInt"),
"test_bool": c.Bool("testBool"),
"test_float": c.Float("testFloat"),
"test_obj": s.Object{
"test_obj_string": c.Str("testObjString"),
},
}
)
func eventMapping(input map[string]interface{}) common.MapStr {
return schema.Apply(input)
}
----

In the above example, note that it is possible to create the schema object once
but to apply to all events.

[float]
=== Configuration File
Expand Down
4 changes: 2 additions & 2 deletions metricbeat/schema/doc.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
/*
Package schema contains helper utilities to parse, convert, rename and reorganize data
Package schema contains helper utilities to parse, convert, rename, and reorganize data
from the format returned by the various monitored systems to the data model used by
Metricbeat.
See the unit tests in the mapstrstr and mapstriface for usage examples.
See the docs in the mapstrstr and mapstriface sub-packages for usage examples.
*/
package schema

0 comments on commit 7bd61bf

Please sign in to comment.