Merge pull request #1 from dedemorton/journalbeat-docs

Full review of files in beats/journalbeat/docs

journalbeat/docs/config-options.asciidoc

@@ -0,0 +1,210 @@
+[id="configuration-{beatname_lc}-options"]
+== Configure inputs
+
+++++
+<titleabbrev>Configure inputs</titleabbrev>
+++++
+
+By default, {beatname_uc} reads log events from the default systemd journals. To
+specify other journal files, set the <<{beatname_lc}-paths,`paths`>> option in
+the +{beatname_lc}.inputs+ section of the +{beatname_lc}.yml+ file. Each path
+can be a directory path (to collect events from all journals in a directory), or
+a file path. For example:
+
+["source","sh",subs="attributes"]
+----
+{beatname_lc}.inputs:
+- paths:
+  - "/dev/log"
+  - "/var/log/messages/my-journal-file.journal"
+----
+
+Within the configuration file, you can also specify options that control how
+{beatname_uc} reads the journal files and which fields are sent to the
+configured output. See <<{beatname_lc}-options>> for a list of available
+options.
+
+The following examples show how to configure {beatname_uc} for some common use
+cases.
+
+[[monitor-multiple-journals]]
+.Example 1: Monitor multiple journals under the same directory
+This example configures {beatname_uc} to read from multiple journals that are
+stored under the same directory. {beatname_uc} merges all journals under the
+directory into a single event stream and reads the events. With `seek` set to
+`cursor`, {beatname_uc} starts reading at the beginning of the journal, but will
+continue reading at the last known position after a reload or restart.
+["source","sh",subs="attributes"]
+----
+{beatname_lc}.inputs:
+- paths: ["/path/to/journal/directory"]
+  seek: cursor
+----
+
+[[filter-using-field-names]]
+.Example 2: Fetch log events for Redis running on Docker (uses field names from systemd)
+This example configures {beatname_uc} to fetch log events for Redis running in a
+Docker container. The fields are matched using field names from the systemd
+journal.
+["source","sh",subs="attributes"]
+----
+{beatname_lc}.inputs:
+- paths: []
+  include_matches:
+    - "CONTAINER_TAG=redis"
+    - "_COMM=redis"
+----
+
+[[filter-using-translated-names]]
+.Example 3: Fetch log events for Redis running on Docker (uses translated field names)
+This example also configures {beatname_uc} to fetch log events for Redis running
+in a Docker container. However, in this example the fields are matched using the
+<<translated-fields,translated field names>> provided by {beatname_uc}.
+["source","sh",subs="attributes"]
+----
+{beatname_lc}.inputs:
+- paths: []
+  include_matches:
+    - "container.image.tag=redis"
+    - "process.name=redis"
+----
+
+[id="{beatname_lc}-options"]
+[float]
+=== Configuration options
+You can specify the following options to configure how {beatname_uc} reads the
+journal files.
+
+[float]
+[id="{beatname_lc}-paths"]
+==== `paths`
+
+A list of paths that will be crawled and fetched. Each path can be a directory
+path (to collect events from all journals in a directory), or a file path. If
+you specify a directory, {beatname_uc} merges all journals under the directory
+into a single journal and reads them.
+
+If no paths are specified, {beatname_uc} reads from the default journal.
+
+[float]
+[id="{beatname_lc}-backoff"]
+==== `backoff`
+
+The number of seconds to wait before trying to read again from journals. The
+default is 1s.
+
+[float]
+[id="{beatname_lc}-max-backoff"]
+==== `max_backoff`
+
+The maximum number of seconds to wait before attempting to read again from
+journals. The default is 60s.
+
+[float]
+[id="{beatname_lc}-seek"]
+==== `seek`
+
+The position to start reading the journal from. Valid settings are:
+
+// REVIEWERS: Not sure if I've gotten this quite right. 
+
+* `head`: Starts reading at the beginning of the file. After a restart,
+{beatname_uc} resends all log messages in the journal.
+* `tail`: Starts reading at the end of the file. After a restart, 
+{beatname_uc} resends the last message, which might result in duplicates. If
+multiple log messages are written to a journal while {beatname_uc} is down,
+only the last log message is sent on restart. 
+* `cursor`: On first read, starts reading at the beginning of the file. After a
+reload or restart, continues reading at the last known position.
+
+If you have old log files and want to skip lines, start {beatname_uc} with
+`seek: tail` specified. Then stop {beatname_uc}, set `seek: cursor`, and restart
+{beatname_uc}.
+
+[float]
+[id="{beatname_lc}-include-matches"]
+==== `include_matches`
+
+A list of filter expressions used to match fields. The format of the expression
+is `field=value`. {beatname_uc} fetches all events that exactly match the
+expressions. Pattern matching is not supported.
+
+To reference fields, use one of the following:
+
+* The field name used by the systemd journal. For example,
+`CONTAINER_TAG=redis` (<<filter-using-field-names,see a full example>>).
+* The <<translated-fields,translated field name>> used by
+{beatname_uc}. For example, `container.image.tag=redis`
+(<<filter-using-translated-names,see a full example>>). {beatname_uc}
+does not translate all fields from the journal. For custom fields, use the name
+specified in the systemd journal.
+
+[float]
+[[translated-fields]]
+=== Translated field names
+
+You can use the following translated names in filter expressions to reference
+journald fields:
+
+[horizontal]
+*Journald field name*:: *Translated name*
+`COREDUMP_UNIT`::             `journald.coredump.unit`
+`COREDUMP_USER_UNIT`::        `journald.coredump.user_unit`
+`OBJECT_AUDIT_LOGINUID`::     `journald.object.audit.login_uid`
+`OBJECT_AUDIT_SESSION`::      `journald.object.audit.session`
+`OBJECT_CMDLINE`::            `journald.object.cmd`
+`OBJECT_COMM`::               `journald.object.name`
+`OBJECT_EXE`::                `journald.object.executable`
+`OBJECT_GID`::                `journald.object.gid`
+`OBJECT_PID`::                `journald.object.pid`
+`OBJECT_SYSTEMD_OWNER_UID`::  `journald.object.systemd.owner_uid`
+`OBJECT_SYSTEMD_SESSION`::    `journald.object.systemd.session`
+`OBJECT_SYSTEMD_UNIT`::       `journald.object.systemd.unit`
+`OBJECT_SYSTEMD_USER_UNIT`::  `journald.object.systemd.user_unit`
+`OBJECT_UID`::                `journald.object.uid`
+`_AUDIT_LOGINUID`::            `process.audit.login_uid`
+`_AUDIT_SESSION`::             `process.audit.session`
+`_BOOT_ID`::                   `host.boot_id`
+`_CAP_EFFECTIVE`::             `process.capabilites`
+`_CMDLINE`::                   `process.cmd`
+`_CODE_FILE`::                 `journald.code.file`
+`_CODE_FUNC`::                 `journald.code.func`
+`_CODE_LINE`::                 `journald.code.line`
+`_COMM`::                      `process.name`
+`_EXE`::                       `process.executable`
+`_GID`::                       `process.uid`
+`_HOSTNAME`::                  `host.name`
+`_KERNEL_DEVICE`::            `journald.kernel.device`
+`_KERNEL_SUBSYSTEM`::         `journald.kernel.subsystem`
+`_MACHINE_ID`::                `host.id`
+`_MESSAGE`::                   `message`
+`_PID`::                       `process.pid`
+`_PRIORITY`::                  `syslog.priority`
+`_SYSLOG_FACILITY`::           `syslog.facility`
+`_SYSLOG_IDENTIFIER`::         `syslog.identifier`
+`_SYSLOG_PID`::                `syslog.pid`
+`_SYSTEMD_CGROUP`::            `systemd.cgroup`
+`_SYSTEMD_INVOCATION_ID`::    `systemd.invocation_id`
+`_SYSTEMD_OWNER_UID`::         `systemd.owner_uid`
+`_SYSTEMD_SESSION`::           `systemd.session`
+`_SYSTEMD_SLICE`::             `systemd.slice`
+`_SYSTEMD_UNIT`::              `systemd.unit`
+`_SYSTEMD_USER_SLICE`::       `systemd.user_slice`
+`_SYSTEMD_USER_UNIT`::         `systemd.user_unit`
+`_TRANSPORT`::                 `systemd.transport`
+`_UDEV_DEVLINK`::             `journald.kernel.device_symlinks`
+`_UDEV_DEVNODE`::             `journald.kernel.device_node_path`
+`_UDEV_SYSNAME`::             `journald.kernel.device_name`
+`_UID`::                       `process.uid`
+
+
+The following translated fields for
+https://docs.docker.com/config/containers/logging/journald/[Docker] are also
+available: 
+
+[horizontal]
+`CONTAINER_ID`::              `conatiner.id_truncated`
+`CONTAINER_ID_FULL`::         `container.id`
+`CONTAINER_NAME`::            `container.name`
+`CONTAINER_PARTIAL_MESSAGE`:: `container.partial`
+`CONTAINER_TAG`::             `container.image.tag`

journalbeat/docs/configuring-howto.asciidoc

@@ -0,0 +1,68 @@
+[id="configuring-howto-{beatname_lc}"]
+= Configuring {beatname_uc}
+
+[partintro]
+--
+
+Before modifying configuration settings, make sure you've completed the
+<<{beatname_lc}-configuration,configuration steps>> in the Getting Started.
+This section describes some common use cases for changing configuration options.
+
+include::../../libbeat/docs/shared-configuring.asciidoc[]
+
+The following topics describe how to configure {beatname_uc}:
+
+* <<configuration-{beatname_lc}-options>>
+* <<configuration-general-options>>
+* <<configuring-internal-queue>>
+* <<configuring-output>>
+* <<configuration-ssl>>
+* <<filtering-and-enhancing-data>>
+* <<configuring-ingest-node>>
+* <<configuration-path>>
+* <<setup-kibana-endpoint>>
+* <<configuration-template>>
+* <<configuration-logging>>
+* <<using-environ-vars>>
+* <<yaml-tips>>
+* <<regexp-support>>
+* <<http-endpoint>>
+* <<{beatname_lc}-reference-yml>>
+
+--
+
+include::./config-options.asciidoc[]
+
+include::./general-options.asciidoc[]
+
+include::../../libbeat/docs/queueconfig.asciidoc[]
+
+include::../../libbeat/docs/outputconfig.asciidoc[]
+
+include::../../libbeat/docs/shared-ssl-config.asciidoc[]
+
+include::./filtering.asciidoc[]
+
+include::../../libbeat/docs/shared-config-ingest.asciidoc[]
+
+include::../../libbeat/docs/shared-path-config.asciidoc[]
+
+include::../../libbeat/docs/shared-kibana-config.asciidoc[]
+
+include::../../libbeat/docs/setup-config.asciidoc[]
+
+include::../../libbeat/docs/loggingconfig.asciidoc[]
+
+:standalone:
+include::../../libbeat/docs/shared-env-vars.asciidoc[]
+:standalone!:
+
+:standalone:
+include::../../libbeat/docs/yaml.asciidoc[]
+:standalone!:
+
+include::../../libbeat/docs/regexp.asciidoc[]
+
+include::../../libbeat/docs/http-endpoint.asciidoc[]
+
+include::../../libbeat/docs/reference-yml.asciidoc[]

journalbeat/docs/faq.asciidoc

@@ -0,0 +1,10 @@
+[[faq]]
+== Frequently asked questions
+
+This section contains frequently asked questions about {beatname_uc}. Also check
+out the https://discuss.elastic.co/c/beats/{beatname_lc}[{beatname_uc}
+discussion forum].
+
+include::../../libbeat/docs/faq-limit-bandwidth.asciidoc[]
+
+include::../../libbeat/docs/shared-faq.asciidoc[]