Syncoid

Syncoid is a replication tool, which facilitates the asynchronous incremental replication of ZFS filesystems. Syncoid allows you to reliably replicate your ZFS snapshots to another ZFS filesystem, near or far.

Using Syncoid
- Options
Examples
Pruning

Primer

Syncoid automates the process of incrementally replicating ZFS datasets to remote hosts. Traditionally, this is accomplished through the use of ZFS send and receive replication. On the first replication, Syncoid transfers the dataset and all snapshots to the remote host. On subsequent replications, Syncoid only transfers the content which have changed since the previous replication (including intermediate snapshots.) Syncoid is agnostic with respect to how the snapshots are created, destroyed, or managed on either host. Syncoid manages it's own snapshots to facilitate replication but these are created and deleted by Syncoid at will and should not be relied on for data retention.

Using Syncoid

Syncoid is easy to use. Simply call syncoid with a target and destination pool, and replication will begin.

syncoid data/images/vm backup/images/vm

This will begin replication from the local dataset data/images/vm to another local dataset backup/images/vm. Syncoid also supports both push and pull replication with remote hosts over ssh. This is accomplished by adding a username and hostname before the dataset you'd like to replicate.

syncoid data/images/vm root@remotehost:backup/images/vm

This would replicate your local dataset (the dataset to be backed up) to a remote dataset (the machine to store your backups).

syncoid root@remotehost:data/images/vm backup/images/vm

This would replicate a remote dataset to a local dataset. In this example, the machine which will be storing your backups is pulling from the machine being backed up.

Options

Syncoid's default behavior is changed using command flags. Unlike sanoid, there is no config file to modify. Here are the options which can be passed to syncoid at runtime.

Flag	Description
[source]	This is the source dataset. It can be either local or remote
[destination]	This is the destination dataset. It can be either local or remote
--identifier=	Adds the given identifier to the snapshot name after "syncoid_" prefix and before the hostname. This enables the use case of reliable replication to multiple targets from the same host. The following chars are allowed: a-z, A-Z, 0-9, _, -, : and . .
-r --recursive	This will also transfer child datasets
--skip-parent	This will skip the syncing of the parent dataset. Does nothing without '--recursive' option
--compress	Currently accepted options: gzip, pigz-fast, pigz-slow, zstd-fast, zstd-slow, lz4, xz, lzo (default) & none. If the selected compression method is unavailable on the source and destination, no compression will be used
--source-bwlimit <limit t\|g\|m\|k>	This is the bandwidth limit in bytes (kbytes, mbytes, etc) per second imposed upon the source. This is mainly used if the target does not have mbuffer installed, but bandwidth limits are desired
--target-bwlimit <limit t\|g\|m\|k>	This is the bandwidth limit in bytes (kbytes, mbytesm etc) per second imposed upon the target. This is mainly used if the source does not have mbuffer installed, but bandwidth limits are desired.
--no-command-checks	Does not check the existence of commands before attempting the transfer, providing administrators a way to run the tool with minimal overhead and maximum speed, at risk of potentially failed replication, or other possible edge cases. It assumes all programs are available, and should not be used in most situations. This is an not an officially supported run mode
--no-stream	This argument tells syncoid to use -i incrementals, not -I. This updates the target with the newest snapshot from the source, without replicating the intermediate snapshots in between. (If used for an initial synchronization, will do a full replication from newest snapshot and exit immediately, rather than starting with the oldest and then doing an immediate -i to the newest.)
--no-sync-snap	This argument tells syncoid to restrict itself to existing snapshots, instead of creating a semi-ephemeral syncoid snapshot at execution time. Especially useful in multi-target (A->B, A->C) replication schemes, where you might otherwise accumulate a large number of foreign syncoid snapshots
--create-bookmark	This argument tells syncoid to create a zfs bookmark for the newest snapshot after it got replicated successfully. The bookmark name will be equal to the snapshot name. Only works in combination with the --no-sync-snap option. This can be very useful for irregular replication where the last matching snapshot on the source was already deleted but the bookmark remains so a replication is still possible
--no-clone-rollback	Do not rollback clones on target
--no-rollback	Do not rollback anything (clones or snapshots) on target host
--exclude=REGEX	The given regular expression will be matched against all datasets which would be synced by this run and excludes them. This argument can be specified multiple times
--no-resume	This argument tells syncoid to not use resumeable zfs send/receive streams
--force-delete	Remove target datasets recursively (WARNING: this will also affect child datasets with matching snapshots/bookmarks), if there are no matching snapshots/bookmarks
--no-clone-handling	This argument tells syncoid to not recreate clones on the target on initial sync and doing a normal replication instead
--dumpsnaps	This prints a list of snapshots during the run
--no-privilege-elevation	Bypass the root check and assume syncoid has the necessary permissions (for use with ZFS permission delegation)
--recvoptions	Additional options for the `zfs recv` command. (e.g. --recvoptions='-x canmount')
--sendoptions	Additional options for the `zfs send` command. (`zfs send -w` equals `--sendoptions="w"`)
--sshport	Allow sync to/from boxes running SSH on non-standard ports
--sshcipher	Instruct ssh to use a particular cipher set
--sshoption	Passes option to ssh. This argument can be specified multiple times
--sshkey	Use specified identity file as per ssh -i
--quiet	Suppress non-error output
--debug	This prints out quite alot of additional information during a sanoid run, and is normally not needed
--help	Show help message
--version	Print version and exit
--monitor-version	This doesn't do anything right now

Examples

The examples assume that:

Your local pool is called rpool
The remote pool is called tank/backup/rpool
Your local user us called localuser
Your remote user is called remoteuser

/usr/sbin/syncoid --recursive rpool HOST:tank/backup/rpool

In this basic example, we're doing push-replication. We pass the --recursive flag as we want every dataset under rpool to also be replicated.

Running without root

In order to run syncoid without root permissions, you'll need to set a few ZFS permissions on both the sending and receiving end.

On the machine you want to sync the dataset from, you'll need the send, hold, mount, snapshot, and destroy permissions. The last three are not needed when using --no-sync-snap. You can apply them as such:

sudo zfs allow -u localuser send,hold,mount,snapshot,destroy rpool

On the receiving side, you will need the compression, mountpoint, create, mount, receive, rollback and destroy permissions. You can apply them as such:

sudo zfs allow -u remoteuser compression,mountpoint,create,mount,receive,rollback,destroy tank/backup/rpool

Now you can run syncoid as your own user, rather than root. This does require the `--no-privilege-elevation option, as shown in the example below:

localuser@localmachine:~$ /usr/sbin/syncoid --no-privilege-elevation --recursive rpool remoteuser@HOST:tank/backup/rpool

Snapshot Management with Sanoid

Syncoid is agnostic to how snapshots are managed on either host. By default, snapshots created on the host are not destroyed on the destination when they are removed from the host. This protects against accidental data loss. It also allows for different data retention policies on the source and destination hosts. If the snapshots were taken with sanoid, you can run sanoid on the remote target with autosnap = no and autoprune = yes to prune old snapshots:

sanoid.conf:

[backup/images/vm]
    ### remove outdated snapshots
    autoprune = yes
    ### don't have sanoid take snapshots as they are coming from syncoid
    autosnap = no
    ### define what to keep
    frequently = 0
    hourly = 36
    daily = 30
    monthly = 6
    yearly = 0

It's possible to create a reusable template as well:

[template_backup]
    ### remove outdated snapshots
    autoprune = yes
    ### don't take snapshots as they are coming from syncoid
    autosnap = no
    ### define what to keep
    frequently = 0
    hourly = 36
    daily = 30
    monthly = 6
    yearly = 0

# same as previous example
[backup/images/vm]
    use_template = backup

# recursively pruning all datasets on backup/images is also possible
[backup/images]
    use_template = backup
    recursive = yes
    ### uncommenting the following line excludes snapshots on "backup/images" itself from pruning
    # process_children_only = yes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly