Live-tail implementation

[smilu97]

Live Tail

The purpose of the live-tail feature is to literally pull the logs that have just been generated from a specific file on the target host up to the web in real time. In order for the user to use the live-tail feature, the telegraf agent's execd output plugin must be used to connect to the grpc agent provided by pinpoint, which will send logs only when needed in response to signals from the pinpoint collector. The logs sent from the pinpoint output are delivered to the web via the redis pub/sub channel.

FileKey Identifier

Every single log files must have their own unique identifier, which is called FileKey

A FileKey consists of three pieces: HostGroupName, HostName, and FileName. The HostGroupName and FileName must be determined by the user, while the HostName is determined by the pinpoint agent, which is the same as the result of /bin/hostname.

The FileKeys are usually serialized with this format, {{HostGroupName}}:{{HostName}}:{{FileName}} in Pinpoint, and it can be found at the redis with the keys below log:files:*

Redis usages

The collector should periodically save the registered FileKeys to the Redis value store so that the web knows which FileKeys are currently available.

Once the web has informed users of the FileKeys available to them, then users can specify which FileKey is associated with the file they want to read, and request corresponding live-tail.

Collectors are always listening to those kind of requests via Redis subscribe, and at least one of the collectors must be able to response to the live-tail request. The response must be sent via Redis publish, and received by web via Redis subscribe also at this time.

Web API Specifications

Getting all HostGroupNames

This API returns all distinct HostGroupNames of FileKeys which is registered to the Pinpoint.

GET /log/hostGroups

Response example

["HostGroupA", "HostGroupB"]

Getting Hosts of a HostGroup

This API returns the FileKeys by the HostGroupName in path variable

GET /log/hostGroups/{{hostGroupName}}/hosts

Response example

[{"name":"HostA","files":["fileA"]}]

Live Tail WebSocket API

This API sends the log in real time through WebSocket connection.

WS /log/liveTail?hostGroupName={{hostGroupName}}&hostName={{hostName}}&fileName={{fileName}}

Response example

{
    "seq": 138,
    "logs": [
        {
            "seq": 229,
            "log": "13:19:31 [Worker-30795] DEBUG i.g.n.WeirdHandler :24650 -- THIS IS GENERATED FAKE LOG"
        }
    ]
}