Input configuration

SuroInput defines how Suro server consumes messages. If the input is thrift server, Suro server will get messages from Suro client thrift library. Likewise, if the input is Kafka consumer, Suro server can pull messages from kafka broker. Suro server can have multiple inputs. Suro input configuration is json file and its example for thrift server is the following. Thrift server configuration itself should be done by server properties.

[
    {
        "type": "thrift"
    }
]

Kafka consumer

For kafka consumer, single SuroInput can consume only one topic. So, if we want to consume multiple topics from kafka broker, we can define as the following:

[
    {
        "type":"kafka",
        "consumerProps":{
            "zookeeper.connect": "zk1:2181",
            "fetch.message.max.bytes": "10000000",
            "group.id": "surorouter-logsummary",
            "consumer.timeout.ms": "1000"
        },
        "topic": "nf_errors_log"
    },
    {
        "type":"kafka",
        "consumerProps":{
            "zookeeper.connect": "zk1:2181",
            "fetch.message.max.bytes": "10000000",
            "group.id": "surorouter-logsummary",
            "consumer.timeout.ms": "1000"
        },
        "topic": "trace_annotation"
    }
]

Kafka consumer needs only two properties, consumerProps and topic. consumerProps is Kafka 0.8 high level consumer properties.