fluentd input/output plugin for kestrel.
fluentd is an event collector system. see github.com/fluent/fluentd
kestrel is a simple, distributed message queue. see github.com/robey/kestrel
gem install fluent-plugin-kestrel
<match kestrel.**> type kestrel host localhost # (required) kestrel host queue fluent # (required) queue name of kestrel port 22133 # (optional) kestrel port. default 22133 ttl 100 # (optional) ttl(sec). default=0 (never expire) # output format options output_include_time false # (optional) default true output_include_tag false # (optional) default true remove_prefix kestrel # (optional) remove tag prefix. # ex) "kestrel.access" -> "access" field_separator COMMA # (optional) default '\t'. # you can use SPACE and COMMA time_format %Y-%m-%d %H-%M-%S # (optional) </match>
<source> type kestrel host localhost # (required) kestrel host queue fluent # (required) kestrel queue name tag kestrel.log # (required) fluentd tag port 22133 # (optional) kestrel port. default 22133 timeout 10 # (optional) default 10. </source>
[time]\t[tag]\t[resource(JSON)] 2012-02-11T16:38:54Z apache.access {"host":"::1","user":"-","method":"GET","path":"/test/}
field_separator COMMA
[time],[tag],[resource(JSON)] 2012-02-11T16:38:54Z,apache.access,{"host":"::1","user":"-","method":"GET","path":"/test/}
field_separator SPACE
[time] [tag] [resource(JSON)] 2012-02-11T16:38:54Z apache.access {"host":"::1","user":"-","method":"GET","path":"/test/}
output_include_tag false
[time]\t[resource(JSON)] 2012-02-11T16:38:54Z {"host":"::1","user":"-","method":"GET","path":"/test/}
output_include_time false
[tag]\t[resource(JSON)] apache.access {"host":"::1","user":"-","method":"GET","path":"/test/}
remove_prefix apache
[time]\t[tag(prefix removed)]\t[resource(JSON)] 2012-02-11T16:38:54Z access {"host":"::1","user":"-","method":"GET","path":"/test/}
time_format %Y/%m/%d %H-%M-%S
[time(original format)]\t[tag]\t[resource(JSON)] 2012/02/11 16-38-54 apache.access {"host":"::1","user":"-","method":"GET","path":"/test/}
you can see its data format like so:
$ telnet localhost 22133 get test VALUE test 0 125 2012-02-11T16:38:54Z apache.access {"host":"::1","user":"-","method":"GET","path":"/test/","code":"304","size":"-"} END
-
patches welcome!
- Copyright
-
Copyright © 2012 Junichiro Takagi
- License
-
Apache License, Version 2.0