Forward logs to Fluentd

Define a LogDevice which wraps FluentLogger

require 'rubygems'
require 'ougai'
require 'fluent-logger'

class FluentLoggerDevice
  def initialize(host = 'localhost', port = 24224, opts = {})
    @log = Fluent::Logger::FluentLogger.new(nil, host: host, port: port)
    @default_tag = opts[:default_tag] || :app
    @tag_field = opts[:tag_field] || :tag
    @time_field = opts[:time_field] || :time
  end
  
  def write(data)
    tag = data.delete(@tag_field) || @default_tag
    time = data.delete(@time_field) || Time.now
    unless @log.post_with_time(tag, data, time)
      p @log.last_error
    end
  end
  
  def close
    @log.close
  end
end

• write method needs to receive a Hash because FlentLogger serializes it as MessagePack.
• Extracting tag and time from a Hash, they are passed to the arguments of FlentLogger#post_with_time method.

An example how to use

Run a fluentd listening localhost:24224 with following configuration.

fluentd.conf

<source>
  @type forward
</source>

<match *.*>
  @type stdout
</match>

Run code

• Define the tag for fluentd in with_fields.
• Bunyan formatter is disabled to dump logs as JSON by jsonize = false. The logs are passed to a fluentd client as they are Hash with that.

logger = Ougai::Logger.new(FluentLoggerDevice.new)
logger.with_fields = { tag: 'app.worker' }
logger.formatter.jsonize = false # formatter is Ougai::Formatters::Bunyan

logger.info('User Info', name: { first: 'Taro', family: 'Suzuki' }, age: 15, height: '170cm')

begin
  raise 'DB Error'
rescue => err
  logger.error('Failed to register a user.', err)
end

logger.warn { 'Ignored error' }

logger.close

Output result

2017-08-18 01:13:03.000000000 +0900 app.worker: {"name":{"first":"Taro","family":"Suzuki"},
"hostname":"mint","pid":34317,"level":30,"v":0,"age":15,
"height":"170cm","msg":"User Info"}
2017-08-18 01:13:03.000000000 +0900 app.worker: {"name":"fluent_test","hostname":"mint",
"pid":34317,"level":50,"v":0,"msg":"Failed to register a user.",
"err":{"name":"RuntimeError","message":"DB Error","stack":"fluent_test.rb:34:in `<main>'"}}
2017-08-18 01:13:03.000000000 +0900 app.worker: {"name":"fluent_test","hostname":"mint",
"pid":34317,"level":40,"v":0,"msg":"Ignored error"}