zfs-snap-diff

#

Index

zfs-snap-diff helps you with your zfs snapshots.

Find file versions

zfs-snap-diff searches different file versions in your zfs snapshots for you.

If you have hundreds or thousands of zfs snapshots, zfs-snap-diff searches through the snapshots and shows you only the snapshots where a given file was modified.

You can inspect a diff from the actual file version to the older file version in the snapshot, revert a single change or restore a whole file.

Management

You can create, destroy, rename, rollback and clone zfs snapshots, use the integrated directory browser to navigate in your snapshots (directory history) and download a directory as a zip-archive.

Simple use

~zfs-snap-diff~ has a web frontend, so it can run on your local work machine or on your remote file / backup server (no Xserver necessary). To keep it portable it’s made as a single static compiled executable.

For a terminal based tool see zsd.

License

zfs-snap-diff is released under the **MIT** License. See the license file for more information.

Installation

Binary packages

You can download the latest binary package from **here** or from the GitHub release page.

{{< hint warning >}} If you use any snapshot management functions, remember to use the -use-sudo flag! {{< /hint >}}

{{< hint info >}} Currently, the tar archive contains only the executables. If you need distribution specific packages, or binaries for any other platform, feel free to contact me. {{< /hint >}}

Build from source

The backend of zfs-snap-diff is implemented in Go, the frontend in PureScript.

`go`

I use go-bindata to decode the frontend code and all dependencies to a go source file so you only need the go compiler to compile it yourself.

The minimum supported go version is go1.12.

clone this repo: git clone --depth 1 https://github.com/j-keck/zfs-snap-diff
cd zfs-snap-diff
build it: go build -ldflags="-X main.version=$(git describe)" ./cmd/zfs-snap-diff

The optional -ldflags="-X main.version=$(git describe)" flag updates the version string in the binary.

`nix`

I use nix to build my projects. The nix build also compiles the frontend to javascript and decodes it in pkg/webapp/bindata.go.

clone this repo: git clone --depth 1 https://github.com/j-keck/zfs-snap-diff
change to the checkout directory: cd zfs-snap-diff
build it: nix-build -A zfs-snap-diff

The build artifacts zfs-snap-diff and zsd are in ./result/bin/.

To crosscompile the binary use:

FreeBSD: nix-build -A zfs-snap-diff --argstr goos freebsd
MacOS: nix-build -A zfs-snap-diff --argstr goos darwin
Solaris: nix-build -A zfs-snap-diff --argstr goos solaris

zfs-snap-diff

zfs-snap-diff - web application to find older file versions in zfs snapshots and zfs snapshot management tool.

With zfs-snap-diff you can

create, destroy, rename, rollback and clone snapshots in the webapp
find older file versions in your zfs-snapshots for a given file
view the file content of a given snapshot
inspect a diff from the older version to the actual version
revert a single change
restore a version from a zfs snapshot
download a file version
browse the directory content of a snapshot
download a zip-archive from any folder in your snapshots
bookmark often used folders

Usage

main⟩ zfs-snap-diff -h
zfs-snap-diff - web application to find older file versions in zfs snapshots and zfs snapshot management tool.

USAGE:
  ./zfs-snap-diff [OPTIONS] <ZFS_DATASET_NAME>

OPTIONS:
  -V	print version and exit
  -a	listen on all interfaces
  -cert string
        TLS certificate file
  -compare-method string
        used method to determine if a file was modified ('auto', 'mtime', 'size+mtime', 'content', 'md5') (default "auto")
  -d int
        days to scan (default 7)
  -diff-context-size int
        show N lines before and after each diff (default 2)
  -key string
        TLS private key file
  -l string
        webserver listen address (default "127.0.0.1")
  -log-locations
        log messages with caller location
  -log-timestamps
        log messages with timestamps in unix format
  -mount-snapshots
        mount snapshot (only necessary if it's not mounted by zfs automatically
  -p int
        webserver port (default 12345)
  -tls
        use TLS - NOTE: -cert <CERT_FILE> -key <KEY_FILE> are mandatory
  -use-cache-dir-for-backups
        use platform depend user local cache directory for backups (default true)
  -use-sudo
        use sudo when executing 'zfs' commands
  -v	debug output
  -vv
        trace output with caller location
  -webapp-dir string
        when given, serve the webapp from the given directory

Project home page: https://j-keck.github.io/zfs-snap-diff

Startup

startup a server instance

./zfs-snap-diff [OPTIONS] <ZFS_DATASET_NAME>

This starts an embedded webserver and serves the included web-app at http://127.0.0.1:12345.

open your webbrowser at

http://127.0.0.1:12345

Browse the actual filesytem

You can browse the actual filesystem and inspect a diff from the actual file version to the older file version in the selected snapshot, revert a single change or restore a whole file.

Browse snapshots

In this view you can view and manage all snapshots.

and inspect the directory content where the snapshot was created

Create snapshots

To create a snapshot of the actual dataset use the camera symbol {{< fas camera >}} in the dataset selector.

You can enter a snapshot name in “Snapshot name template” and zfs-snap-diff will show the resulting name in “Snapshot name”.

The template supports the following format sequences:

Format sequences are alike the `date` command
  %d: day of month (e.g., 01)
  %m: month (01..12)
  %y: last two digits of year (00..99)
  %Y: year
  %F: full date; like %Y-%m-%d
  %H: hour (00..23)
  %I: hour (01..12)
  %M: minute (00..59)
  %S: second (00..60)
  %s: seconds since 1970-01-01 00:00:00 UTC

The default snapshot name template is per ~snapshot-name-template~ configurable.

Snapshot management

{{< hint warning >}} If you use any snapshot management functions, remember to use the -use-sudo flag! {{< /hint >}}

You can click the {{< fas ellipsis-v >}} symbol to show the snapshots actions.

Rename snapshot

Destroy snapshot

Clone snapshot

file:/images/browse-snapshots-clone.png

{{< hint warning >}} The newly created dataset will only listed if the parent datasets mountpoint is **not** none or legacy. {{< /hint >}}

Rollback snapshot

file:/images/browse-snapshots-rollback.png

Download zip-archive

With the {{< fas file-archive >}} symbol in the file browser you can download a whole directory as a zip-archive. You can download a archive from the actual filesystem or from a snapshot.

The archive size is restricted by default. You can configure per ~max-archive-unpacked-size-mb~.

zsd

{{< hint warning >}} zsd is now a independent project. You find it here: https://j-keck.github.io/zsd.

The next version of zfs-snap-diff comes without zsd. {{< /hint >}}

zsd - cli tool to find older versions of a given file in your zfs snapshots.

With zsd you can

find older file versions in your zfs snapshots for a given file
view the file content from a given snapshot
inspect a diff from the older version to the actual version
restore a version from a zfs snapshot

It uses the same code as zfs-snap-diff to find different file versions in your zfs snapshots.

Usage

main⟩ zsd -h
zsd - cli tool to find older versions of a given file in your zfs snapshots.

USAGE:
 ./zsd [OPTIONS] <FILE> <ACTION>

OPTIONS:
  -H	Scripting mode. Do not print headers, print absolute dates and separate fields by a single tab
  -V	print version and exit
  -d int
        days to scan (default 2)
 -mount-snapshots
        mount snapshot (only necessary if it's not mounted by zfs automatically)
 -snapshot-timemachine
        Special output for Snapshot-timemachine (https://github.com/mrBliss/snapshot-timemachine)
 -use-sudo
        use sudo when executing 'zfs' commands
  -v	debug output
  -vv
        trace output with caller location

ACTIONS:
  list                : list zfs snapshots where the given file was modified
  cat     <#|SNAPSHOT>: show file content of the given snapshot
  diff    <#|SNAPSHOT>: show a diff from the selected snapshot to the actual version
  restore <#|SNAPSHOT>: restore the file from the given snapshot

You can use the snapshot number from the `list` output or the snapshot name to select a snapshot.

Project home page: https://j-keck.github.io/zfs-snap-diff

List snapshots

Use the list action to list all snapshots where the given file was modified.

main⟩ zsd go.mod list
scan the last 7 days for other file versions
  # | Snapshot                               | Snapshot age
-----------------------------------------------------------
  0 | zfs-auto-snap_hourly-2020-02-12-12h00U | 5 hours
  1 | zfs-auto-snap_hourly-2020-02-12-09h00U | 8 hours

Show file content

Use the cat action to show the file content from the given snapshot.

{{< hint info >}} You can use the snapshot number from the list output or the snapshot name to select a snapshot. {{< /hint >}}

main⟩ zsd go.mod cat 0
module github.com/j-keck/zfs-snap-diff

require (
	 github.com/j-keck/go-diff v1.0.0
	 github.com/j-keck/plog v0.5.0
	 github.com/stretchr/testify v1.4.0 // indirect
)

go 1.12

Show diff

To show a diff from the selected snapshot to the actual version use the diff action.

{{< hint info >}} You can use the snapshot number from the list output or the snapshot name to select a snapshot. {{< /hint >}}

main⟩ zsd go.mod diff 0
Diff from the actual version to the version from: 2020-02-12 10:07:44.434355182 +0100 CET
module github.com/j-keck/zfs-snap-diff

require (
   github.com/BurntSushi/toml v0.3.1
   github.com/j-keck/go-diff v1.0.0
-  github.com/j-keck/plog v0.5.0
+  github.com/j-keck/plog v0.6.0
   github.com/stretchr/testify v1.4.0 // indirect
)

go 1.12

Restore file

To restore a given file with an older version use restore.

{{< hint info >}} You can use the snapshot number from the list output or the snapshot name to select a snapshot. {{< /hint >}}

main⟩ zsd go.mod restore 0
backup from the actual version created at: /home/j/.cache/zfs-snap-diff/backups/home/j/prj/priv/zfs-snap-diff/go.mod_20200212_182709%
version restored from snapshot: zfs-auto-snap_hourly-2020-02-12-12h00U

{{< hint warning >}} A backup of the current version will be created. {{< /hint >}}

Configuration

zfs-snap-diff loads it’s configuration from:

$XDG_CONFIG_HOME/.config/zfs-snap-diff/zfs-snap-diff.toml
$HOME/.config/zfs-snap-diff/zfs-snap-diff.toml

$HOME/Library/Application Support/zfs-snap-diff/zfs-snap-diff.toml

if it does not find a configuration, it will create the following default configuration:

use-cache-dir-for-backups = true
days-to-scan = 2
max-archive-unpacked-size-mb = 200
snapshot-name-template = "zfs-snap-diff-%FT%H:%M"
compare-method = "auto"
diff-context-size = 5

[webserver]
  listen-ip = "127.0.0.1"
  listen-port = 12345
  use-tls = false
  cert-file = ""
  key-file = ""

[zfs]
  use-sudo = false
  mount-snapshots = false

`use-cache-dir-for-backups`

If it’s set to true, the file backups will be stored in the users cache-directory.

On Unix systems, it returns $XDG_CACHE_HOME as specified by https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html if non-empty, else $HOME/.cache. On Darwin, it returns $HOME/Library/Caches. On Windows, it returns %LocalAppData%. On Plan 9, it returns $home/lib/cache.

https://golang.org/pkg/os/#UserCacheDir

If it’s false, it will create the backup file under the actual directory in the ./zfs-snap-diff/ folder.

`days-to-scan`

To speedup the scan for other file versions, zfs-snap-diff performs the scan incremental when you request an older file version. This parameter determines how many days are scanned if you request a older versions.

`max-archive-unpacked-size-mb`

The maximal (unpacked) archive size is restricted by default. Set this to -1 to allow disable this restriction.

`snapshot-name-template`

Snapshot name template. Used to create snapshots in the web-app. The template supports the following format sequences:

Format sequences are alike the `date` command
  %d: day of month (e.g., 01)
  %m: month (01..12)
  %y: last two digits of year (00..99)
  %Y: year
  %F: full date; like %Y-%m-%d
  %H: hour (00..23)
  %I: hour (01..12)
  %M: minute (00..59)
  %S: second (00..60)
  %s: seconds since 1970-01-01 00:00:00 UTC

`compare-method`

Used compare method to find different file versions. This is used when scanning the zfs snapshots to determine if a file was modified in a snapshot.

auto

Uses md5 for text files and size+mtime for others

size

If two files versions have the same filesize, it’s interpreted as the same version.

mtime

If two files versions have the same mtime, it’s interpreted as the same version.

size+mtime

If two files versions have the same size AND mtime, it’s interpreted as the same version.

content

If two files versions have the same content, it’s interpreted as the same version.

md5

If two files versions have the same md5 sum, it’s interpreted as the same version.

`diff-context-size`

Diff context size in the webui.

Changelog

1.1.3

no changes in the application, only in the build-pipeline (note to myself: always separate the app from the pipelines)

all commits from v1.1.2 to v1.1.3

1.1.2

bump deps

all commits from v1.1.1 to v1.1.2

1.1.1

This release contains only changes for ~zsd~.

zsd: new flag -H for scripting mode output.
zsd: new flag -snapshot-timemachine to support mrBliss/snapshot-timemachine

The release / packaging process for the two programms is currently not separated, so i make a “bugfix” release for this changes.

all commits from v1.1.0 to v1.1.1

1.1.0

add snapshot management functions (see docs)
- rename
- destroy
- clone
- rollback
handle keyboard events in input fields
- ‘Enter’ for ‘Submit’
- ‘Esc’ for ‘Cancel’ / close modal
update npm deps

all commits from v1.0.1…v1.1.0

1.0.1

fix destroy snapshot

all commits from v1.0.0…v1.0.1

1.0.0

{{< hint info >}} This version is a complete rewrite.

The backend is implemented in Go (as before) and the frontend in PureScript. {{< /hint >}}

create and destroy snapshots from the webapp
download a complete directory as a zip-archive
~zsd~ cli tool to find different file-versions in the command line
- does not need a running zfs-snap-diff instance
date-range based search for file versions
- this speeds up the scan dramatically if there are thousands snapshots on spinning disks
bookmark support
- bookmarks are per dataset and stored in the browser (Web storage).
works now also with ‘legacy’ mountpoints

all commits from 0.0.10…v1.0.0

0.0.10

use relative url for service endpoints
- to use zfs-snap-diff behind a reverse proxy
- minimal example config snipped for nginx:
  location zfs-snap-diff { proxy_pass http://localhost:12345/; }
optional tls encryption
listen address per ‘-l’ flag configurable

all commits from 0.0.9…0.0.10

0.0.9

show file size and modify timestamp in the file-browser
list directories at first in the file-browser
sortable columns in the file-browser
only regular files / directories are clickable

all commits from 0.0.8…0.0.9

0.0.8

dataset selectable in ‘browse-actual’ view
add size informations to dataset (to match ‘zfs list’ output)
small fixes
code cleanup

all commits from 0.0.7…0.0.8

0.0.7

support sub zfs filesystems (datasets)
optional use sudo when execute zfs commands
- necessary under linux when running as non root
- needs sudo rules
- start `zfs-snap-diff` with-‘-use-sudo’
new view for server messages

all commits from 0.0.6…0.0.7

0.0.6

check if file in snapshot has changed filetype depend:
- text files: md5
- others: size+modTime
diffs created in the backend (per go-diff)
- different presentation: intext / sid- by side
- possibility to revert single changes

all commits from 0.0.5…0.0.6

0.0.5

file compare method configurable: size+modTime (default) or md5
optional limit how many snapshots are scan to search older file version
autohide notifications in frontend
show message if no snapshots found

all commits from 0.0.4…0.0.5

0.0.4

view, diff, download or restore file from a snapshot
view file with syntax highlight
browse old snapshot versions
easy switch “versions” per ‘Older’ / ‘Newer’ buttons
cleanup frontend
refactor backend

all commits 0.0.3…0.0.4

0.0.3

show server errors on frontend
show waiting spinner when loading

all commits 0.0.2…0.0.3

0.0.2

partial frontend configuration from server
fix firefox ui

all commits 0.0.1…0.0.2

0.0.1

prototype

Contact / Support

Contact

{{< columns >}} {{< fas envelope lg >}} Check my GitHub Profile for my mail address. <—> {{< fab twitter lg >}} Send me an direct message on twitter. <—> {{< fab keybase lg >}} Use keybase to contact me. {{< /columns >}}

Support

If you have any questions, trouble or other input, feel free to contact me directly (see Contact) or open a issue@github.

Files

site.org

Latest commit

History

site.org

File metadata and controls

zfs-snap-diff

Index

Find file versions

Management

Simple use

License

Installation

Binary packages

Build from source

go

nix

zfs-snap-diff

Usage

Startup

Browse the actual filesytem

Browse snapshots

Create snapshots

Snapshot management

Rename snapshot

Destroy snapshot

Clone snapshot

Rollback snapshot

Download zip-archive

zsd

Usage

List snapshots

Show file content

Show diff

Restore file

Configuration

use-cache-dir-for-backups

days-to-scan

max-archive-unpacked-size-mb

snapshot-name-template

compare-method

auto

size

mtime

size+mtime

content

md5

diff-context-size

Changelog

1.1.3

1.1.2

1.1.1

1.1.0

1.0.1

1.0.0

0.0.10

0.0.9

0.0.8

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

0.0.1

Contact / Support

Contact

Support

`go`

`nix`

`use-cache-dir-for-backups`

`days-to-scan`

`max-archive-unpacked-size-mb`

`snapshot-name-template`

`compare-method`

`diff-context-size`