Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(parsers.avro): Do not force addition of timestamp as a field #13856

Merged
merged 3 commits into from
Sep 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 1 addition & 19 deletions plugins/parsers/avro/parser.go
Original file line number Diff line number Diff line change
Expand Up @@ -172,24 +172,6 @@ func (p *Parser) createMetric(data map[string]interface{}, schema string) (teleg
// If you have specified your fields in the config, you
// get what you asked for.
fieldList = p.Fields

// Except...if you specify the timestamp field, and it's
// not listed in your fields, you'll get it anyway.
// This will randomize your field ordering, which isn't
// ideal. If you care, list the timestamp field.
if p.Timestamp != "" {
// quick list-to-set-to-list implementation
fieldSet := make(map[string]bool)
for k := range fieldList {
fieldSet[fieldList[k]] = true
}
fieldSet[p.Timestamp] = true
var newList []string
for s := range fieldSet {
newList = append(newList, s)
}
fieldList = newList
}
} else {
for k := range data {
// Otherwise, that which is not a tag is a field
Expand Down Expand Up @@ -255,7 +237,7 @@ func (p *Parser) createMetric(data map[string]interface{}, schema string) (teleg
}
var timestamp time.Time
if p.Timestamp != "" {
rawTime := fmt.Sprintf("%v", fields[p.Timestamp])
rawTime := fmt.Sprintf("%v", data[p.Timestamp])
var err error
timestamp, err = internal.ParseTimestamp(p.TimestampFormat, rawTime, nil)
if err != nil {
Expand Down
2 changes: 1 addition & 1 deletion plugins/parsers/avro/testdata/json-format/message.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@
"up_time": 1166984904,
"cpu_utilization": 14.0,
"memory_utilization": 20.0
}
}
4 changes: 2 additions & 2 deletions plugins/parsers/avro/testdata/json-format/telegraf.conf
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
avro_format = "json"
avro_measurement = "Switch"
avro_tags = ["switch_wwn"]
avro_fields = ["up_time", "cpu_utilization", "memory_utilization"]
avro_fields = ["up_time", "cpu_utilization", "memory_utilization", "statistics_collection_time"]
avro_timestamp = "statistics_collection_time"
avro_timestamp_format = "unix_ms"
avro_schema = '''
Expand All @@ -22,4 +22,4 @@
{"name": "memory_utilization", "type": "float", "default": 0, "doc": "Memory Utilization in %"}
]
}
'''
'''
Original file line number Diff line number Diff line change
@@ -1 +1 @@
measurement,tag=test_tag field=19i,timestamp=1664296121000000i 1664296121000000
measurement,tag=test_tag field=19i 1664296121000000
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@
avro_measurement = "measurement"
avro_tags = [ "tag" ]
avro_fields = [ "field" ]
avro_timestamp = "timestamp"
avro_timestamp_format = "unix_us"
srebhan marked this conversation as resolved.
Show resolved Hide resolved
avro_schema = '''
{
"type":"record",
Expand All @@ -19,10 +17,6 @@
{
"name":"field",
"type":"long"
},
{
"name":"timestamp",
"type":"long"
}
]
}
Expand Down