LeafHeap

Logstash compatible feeder (redis to Elasticsearch). Tailored for our usage at fotopedia.

It was born after we found out about Kibana and wanted a more ligthweight approach than using Logstash. It's a followup of logstash_light and logstash_benchmark (in this benchmark the current implementation reads 12000 events/second on octplane@github.devbox).

This specific implementation reads one or several redis lists and injects them in an ElasticSearch. LeafHeap runs forever and barely complains.

Build

The build system uses sbt.

./sbt assembly

Building

Assuming you have a working Java 1.7 setup:

./sbt assembly

Usage

java -jar target/scala-2.10/LeafHeap-assembly-0.1-SNAPSHOT.jar --help
 --help        : Print help
 --logback VAL : overrides -d
 -d            : run with debug logging parameters
 -e VAL        : ElasticSearch HTTP Url
 -l VAL        : Redis Queues, comma separated list
 -p N          : Redis port
 -r VAL        : Redis hostname

java -jar LeafHeap-assembly-0.1-SNAPSHOT.jar \
 -e http://es.server.internal:9201/ \
 -l master,staging,testing \
 -r redis.server.internal

Credits

From Fotonauts:

• Pierre Baillet @octplane.
• kali

Copyright (c) 2013 Fotonauts released under the MIT license