RemotePrefixFormatter

When a remote file sink such as S3FileSink uploads a file from the local file system, it may need to add some prefix such as directory name. We can configure this prefix with RemotePrefixFormatter.

###DynamicPrefixFormatter: "type":"dynamic"

{
    "type": "S3FileSink",
    ...
    "prefixFormatter": {
        "type": "dynamic",
        "format": "date(yyyyMMdd);static(static_prefix)"
    }
}

Multiple formats can be connected with delimiter semicolon(;). From the above example, prefix will be 20150119/static_prefix.

###DateRegionStackFormatter: "type":"DateRegionStack" This adds <current date>/<region>/<stack> as the prefix of the file name. Current date will be formatted by the property "date". Region and stack can be injected or specified through JSON.