Continuously fetch records from Kinesis Stream and print it. Useful for debugging Kinesis.
$ go get github.com/suzuken/kinesis-tail
$ kinesis-tail -stream your-kinesis-stream-name -region ap-northeast-1 -max-item-size=512
Your Stream information: {
StreamDescription: {
HasMoreShards: false,
RetentionPeriodHours: 24,
Shards: [{
HashKeyRange: {
EndingHashKey: "340282366920938463463374607431768211455",
StartingHashKey: "0"
},
SequenceNumberRange: {
StartingSequenceNumber: "49556705040321186153938184833717233207750712554199973890"
},
ShardId: "shardId-000000000000"
}],
StreamARN: "arn:aws:kinesis:ap-northeast-1:xxxxxxxxxxxx:stream/your-kiness-stream-name",
StreamName: "your-kiness-stream-name",
StreamStatus: "ACTIVE"
}
}
{
ShardIterator: "AAAAAAAAAAF6jxGy30eyuTD4GzFrqAwEUFdWMIiWtFqpzGIShgXfo40VHDGHzNAdN0uvK5j6llWkEJQtdKZC3kQTIOQ/Y7Zj/lyBrV4GTkqz/hT+m/8TCZHU61fbEqU7Wd9V+Wa9szmeBQfdN5BAZDybhSUctjJRT+mgfbxVPXOjLkIrBNpAvzPFLRRaNSJj/ZwxIcG3k3nfwY7yUrcNb42OR6Cu7o1P"
}
ApproximateArrivalTimestamp: 2015-12-01 02:14:23 +0000 UTC
Data: {"id":"5310e2ef-25e3-4942-87df-2aed28320b99","hoge":"fuga"}
SequenceNumber: 49556846757453585699518614739447433851727187225060835330
ApproximateArrivalTimestamp: 2015-12-01 02:24:41 +0000 UTC
Data: {"id":"75ba2e4a-a68d-4d50-89cf-8b954a115a08","hoge":"fuga"}
SequenceNumber: 49556846757453585699518614739460732035742990683338702850
ApproximateArrivalTimestamp: 2015-12-01 04:10:58 +0000 UTC
Data: {"id":"0d669fb6-045e-4299-946c-5786037ab638","hoge":"fuga"}
...
Usage of kineis-tail:
-f tailing kinesis stream forever or not (like: tail -f) (default true)
-interval duration
seconds for waiting next GetRecords request. (default 3s)
-iterator-type string
iterator type. Choose from TRIM_HORIZON(default), AT_SEQUENCE_NUMBER, AT_TIMESTAMP or LATEST. (default "TRIM_HORIZON")
-limit int
limit records length of each GetRecords request. (default 100)
-max-item-size int
max byte size per item for printing. (default 4096)
-region string
the AWS region where your Kinesis Stream is. (default "ap-northeast-1")
-start-time string
timestamp to start reading. only enable when iterator type is AT_TIMESTAMP. acceptable format is YYYY-MM-DDThh:mm:ss.sssTZD (RFC3339 format)
-stream string
your stream name (default "your-stream")
- At first, describe stream.
- Get shard iterator.
GetRecords
iteratively.
MIT
Kenta Suzuki (a.k.a suzuken)