diff --git a/website/source/docs/commands/snapshot.html.markdown b/website/source/docs/commands/snapshot.html.markdown index 1c8f689ac967..2c14643726c6 100644 --- a/website/source/docs/commands/snapshot.html.markdown +++ b/website/source/docs/commands/snapshot.html.markdown @@ -29,6 +29,7 @@ Usage: consul snapshot [options] [args] Subcommands: + agent Periodically saves snapshots of Consul server state inspect Displays information about a Consul snapshot file restore Restores snapshot of Consul server state save Saves snapshot of Consul server state @@ -37,6 +38,7 @@ Subcommands: For more information, examples, and usage about a subcommand, click on the name of the subcommand in the sidebar or one of the links below: +- [agent] (/docs/commands/snapshot/agent.html) (Consul Enterprise only) - [inspect] (/docs/commands/snapshot/inspect.html) - [restore](/docs/commands/snapshot/restore.html) - [save](/docs/commands/snapshot/save.html) @@ -68,5 +70,11 @@ Term 2 Version 1 ``` +To run a daemon process that periodically saves snapshots (Consul Enterprise only): + +``` +$ consul snapshot agent +``` + For more examples, ask for subcommand help or view the subcommand documentation by clicking on one of the links in the sidebar. diff --git a/website/source/docs/commands/snapshot/agent.html.markdown.erb b/website/source/docs/commands/snapshot/agent.html.markdown.erb new file mode 100644 index 000000000000..d01afab04235 --- /dev/null +++ b/website/source/docs/commands/snapshot/agent.html.markdown.erb @@ -0,0 +1,209 @@ +--- +layout: "docs" +page_title: "Commands: Snapshot Agent" +sidebar_current: "docs-commands-snapshot-agent" +--- + +# Consul Snapshot Agent + +Command: `consul snapshot agent` + +~> The [`agent`](/docs/commands/snapshot/agent.html) subcommand described here is + available only in [Consul Enterprise](https://www.hashicorp.com/consul.html) + version 0.7.1 and later. All other [snapshot subcommands](/docs/commands/snapshot.html) + are available in the open source version of Consul. + +The `snapshot agent` subcommand starts a process that takes snapshots of the +state of the Consul servers and saves them locally, or pushes them to an +optional remote storage service. This subcommand is only available in Consul +Enterprise 0.7.1 and later. + +The agent can be run as a long-running daemon process or in a one-shot mode +from a batch job, based on the [`-interval`](#interval) argument. Snapshotting +a remote datacenter is only available in one-shot mode. + +As a long-running daemon, the agent will perform a leader election so multiple +processes can be run in a highly available fashion with automatic failover. The +agent will also register itself with Consul as a service, along with health +checks that show the agent is alive ("Consul Snapshot Agent Alive") and able to +take snapshots ("Consul Snapshot Agent Saving Snapshots"). The latter check is +only added on agents who have become a leader, so it's possible for operators to +tell which instances are alive and on standby and which instance has become +leader and starting saving snapshots. + +As snapshots are saved, they will be reported in the log produced by the agent: + +``` +2016/11/16 21:21:13 [INFO] Snapshot agent running +2016/11/16 21:21:13 [INFO] Waiting to obtain leadership... +2016/11/16 21:21:13 [INFO] Obtained leadership +2016/11/16 21:21:13 [INFO] Saved snapshot 1479360073448728784 +``` + +The number shown with the saved snapshot is its ID, which is based on a UNIX +timestamp with nanosecond resolution, so collisions are unlikely and IDs are +monotonically increasing with time. This makes it easy to locate the latest +snapshot, even if the log data isn't available. The snapshot ID always appears +in the file name when using local storage, or in the object key when using +remote storage. + +Snapshots can be restored using the +[`consul snapshot restore`](/docs/commands/snapshot/restore.html) command, or +the [HTTP API](/docs/agent/http/snapshot.html). + +If ACLs are enabled, a management token must be supplied in order to perform +snapshot operations. + +## Usage + +Usage: `consul snapshot agent [options]` + +#### API Options + +<%= partial "docs/commands/http_api_options" %> + +#### Config File Options: + +* `-config-dir` - Directory to look for JSON config files. Files will be read in + alphabetical order and must end with the extension ".json". This won't + recursively descend directories. This can be specified multiple times on the + command line. + +* `-config-file` - File to read JSON configuration from. Files must end with the + extension ".json". This can be specified multiple times on the command line. + + Config files referenced using `-config-dir` and `-config-file` have the following + format (shown populated with default values): + +```javascript + { + "snapshot_agent": { + "http_addr": "127.0.0.1:8500", + "token": "", + "datacenter": "", + "log": { + "level": "INFO", + "enable_syslog": false, + "syslog_facility": "LOCAL0" + }, + "snapshot": { + "interval": "1h", + "retain": 30, + "stale": false, + "service": "consul-snapshot", + "deregister_after": "72h", + "lock_key": "consul-snapshot/lock", + "max_failures": 3 + }, + "local_storage": { + "path": "." + }, + "aws_storage": { + "access_key_id": "", + "secret_access_key": "", + "s3_region": "", + "s3_bucket": "", + "s3_key_prefix": "consul-snapshot" + } + } + } +``` + +All fields are optional, and config files without a `snapshot_agent` object will +be ignored. At least one config file needs to have a `snapshot_agent` object, or the +snapshot agent will fail to start. The Consul agent is set up to ignore any +`snapshot_agent` object, so it's safe to use common config directories for both agents +if desired. + +#### Snapshot Options + +* `-interval` - Interval at which to perform snapshots + as a time with a unit suffix, which can be "s", "m", "h" for seconds, minutes, + or hours. If 0 is provided, the agent will take a single snapshot and then exit, + which is useful for running snapshots via batch jobs. Defaults to "1h" + +* `-lock-key` - A prefix in Consul's key-value store used to coordinate between + different instances of the snapshot agent order to only have one active instance + at a time. For highly available operation of the snapshot agent, simply run + multiple instances. All instances must be configured with the same lock key in + order to properly coordinate. Defaults to "consul-snapshot/lock". + +* `-max-failures` - Number of snapshot failures after which the snapshot agent + will give up leadership. In a highly available operation with multiple snapshot + agents available, this gives another agent a chance to take over if an agent + is experiencing issues, such as running out of disk space for snapshots. + Defaults to 3. + +* `-retain` - Number of snapshots to retain. After each snapshot is taken, the + oldest snapshots will start to be deleted in order to retain at most this many + snapshots. If this is set to 0, the agent will not perform this and snapshots + will accumulate forever. Defaults to 30. + + +#### Agent Options + +* `-deregister-after` - An interval, after which if the agent is unhealthy it will be + automatically deregistered from Consul service discovery. This is a time with a + unit suffix, which can be "s", "m", "h" for seconds, minutes, or hours. If 0 is + provided, this will be disabled. Defaults to "72h". + +* `-log-level` - Controls verbosity of snapshot agent logs. Valid options are + "TRACE", "DEBUG", "INFO", "WARN", "ERR". Defaults to "INFO". + +* `-service` - The service name to used when registering the agent with Consul. + Registering helps monitor running agents and the leader registers an additional + health check to monitor that snapshots are taking place. Defaults to + "consul-snapshot". + +* `-syslog` - This enables forwarding logs to syslog. Defaults to false. + +* `-syslog-facility` - Sets the facility to use for forwarding logs to syslog. + Defaults to "LOCAL0". + +#### Local Storage Options + +* `-local-path` - Location to store snapshots locally. The default behavior + of the snapshot agent is to store snapshots locally in this directory. Defaults + to "." to use the current working directory. If an alternate storage option is + configured, then local storage will be disabled and this option will be ignored. + +#### Amazon S3 Storage Options + +* `-aws-access-key-id` and `-aws-secret-access-key` - These arguments supply + authentication information for connecting to AWS. These may also be supplied using + the following alternative methods:
+ - `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables + - A credentials file (`~/.aws/credentials` or the file at the path specified by the + `AWS_SHARED_CREDENTIALS_FILE` environment variable) + - ECS task role metadata (container-specific) + - EC2 instance role metadata + +* `-aws-s3-bucket` - S3 bucket to use. Required for S3 storage, and setting this + disables local storage. + +* `-aws-s3-key-prefix` - Prefix to use for snapshot files in S3. Defaults to + "consul-snapshot". + +* `-aws-s3-region` - S3 region to use. Required for S3 storage. + +## Examples + +Running the agent with no arguments will run a long-running daemon process that will +perform leader election for highly available operation, register itself with Consul +service discovery with health checks, take snapshots every hour, retain the last 30 +snapshots, and save snapshots into the current working directory: + +``` +$ consul snapshot agent +``` + +To run a one-shot backup, set the backup interval to 0. This will run a single snapshot +and delete any old snapshots based on the retain settings, but it will not perform any +leader election or service registration: + +``` +$ consul snapshot agent -interval=0 +``` + +Please see the [HTTP API](/docs/agent/http/snapshot.html) documentation for +more details about snapshot internals. diff --git a/website/source/layouts/docs.erb b/website/source/layouts/docs.erb index 726ce44f0d68..defdc57c2000 100644 --- a/website/source/layouts/docs.erb +++ b/website/source/layouts/docs.erb @@ -152,6 +152,9 @@ > snapshot