Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added info about schema to the Metricbeat dev guide #2057

Merged
merged 1 commit into from
Jul 19, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions metricbeat/docs/developer-guide/create-metricset.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,73 @@ For this kind of data, you can use the following `Fetch` interface:
The only difference between this and the previous example is that the second example returns `[]common.MapStr`.
Metricbeat will add the same timestamp to all the events in the list to make it possible to correlate the events.

[float]
==== Parsing and normalizing fields

In Metricbeat we aim to normalize the metric names from all Metricsets to
respect a common {libbeat}/event-conventions.html[set of conventions]. This
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, so this is now also done (conventions). Thx.

makes it easy for users to find and interpret metrics. To simplify parsing,
converting, renaming, and restructuring of the object read from the monitored
system to the Metricbeat format, we have created the
https://godoc.org/github.com/elastic/beats/metricbeat/schema[schema] package
that allows to declaratively define transformations.

For example, assuming this input object:

[source,go]
----
input := map[string]interface{}{
"testString": "hello",
"testInt": "42",
"testBool": "true",
"testFloat": "42.1",
"testObjString": "hello, object",
}
----

And the requirement to transform it into this one:

[source,go]
----
common.MapStr{
"test_string": "hello",
"test_int": int64(42),
"test_bool": true,
"test_float": 42.1,
"test_obj": common.MapStr{
"test_obj_string": "hello, object",
},
}
----

You can use the following code to make the transformations:

[source,go]
----
import (
s "github.com/elastic/beats/metricbeat/schema"
c "github.com/elastic/beats/metricbeat/schema/mapstrstr"
)

var (
schema = s.Schema{
"test_string": c.Str("testString"),
"test_int": c.Int("testInt"),
"test_bool": c.Bool("testBool"),
"test_float": c.Float("testFloat"),
"test_obj": s.Object{
"test_obj_string": c.Str("testObjString"),
},
}
)

func eventMapping(input map[string]interface{}) common.MapStr {
return schema.Apply(input)
}
----

In the above example, note that it is possible to create the schema object once
but to apply to all events.

[float]
=== Configuration File
Expand Down
4 changes: 2 additions & 2 deletions metricbeat/schema/doc.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
/*
Package schema contains helper utilities to parse, convert, rename and reorganize data
Package schema contains helper utilities to parse, convert, rename, and reorganize data
from the format returned by the various monitored systems to the data model used by
Metricbeat.

See the unit tests in the mapstrstr and mapstriface for usage examples.
See the docs in the mapstrstr and mapstriface sub-packages for usage examples.
*/
package schema