Network Block Device (NBD) Mode

NBD Mode

In version 2.0.0 NBD mode was added to s3backer (Note: you should use the latest version to avoid bugs).

NBD mode uses the Network Block Device (NBD) function to provide a "disk" image as opposed to the FUSE function. This is a more natural way for s3backer to work because it avoids the "extra hop" with FUSE requiring a loopback mount.

NBD mode is only supported on Linux.

How Do I Use It?

Quick summary:

1. Add an --nbd flag to the command line and replace the mount point with a /dev/nbdX device file.
2. Create/mount your filesystem on /dev/nbdX.
3. When you're all done, unmount the filesystem and kill s3backer and it will clean up the NBD server it created.

There's more info in the relevant sections of the man page:

SYNOPSIS
     s3backer --nbd [options] bucket[/subdir] /dev/nbdX

DESCRIPTION

   NBD Plugin
     On platforms with ndbkit(1), s3backer is also installed as an nbdkit(1) plugin.

     When using the NBD plugin, command line flags like --blockSize=SIZE should instead be specified as plugin parameters like
     blockSize=SIZE; for boolean flags, specify a value of true.  The bucket name (with optional subdirectory) must be specified
     with the bucket parameter.

     To avoid conflicts with other nbdkit(1) flags, any s3backer flag can be specified with a s3b_ prefix, for example
     s3b_force=true.

     As an alternative to launching nbdkit(1) directly, the --nbd flag causes s3backer to function as a wrapper process for managing
     an NBD session; see below.

OPTIONS

     --nbd   Create a Network Block Device (NBD) instead of a FUSE filesystem.  This is probably a more logical and direct way to do
             things on systems that support NBD.

             s3backer will start up appropriate instances of nbdkit(1) and nbd-client(1) in order to attach the S3 volume to the
             specified device.

             When invoked in this mode, only --doubleDash style flags are supported, with the exception of -f and -d; any other
             flags (including any FUSE-specific flags) are disallowed.  The --nbd flag may not appear in a --configFile file.

             To detach the S3 volume, just kill the s3backer process and it will automatically disconnect the NBD client and shut
             down the NBD server.

     --nbd-flag=string

     --nbd-param=string
             Add string as an nbdkit(1) command line parameter, either before or after the plugin name (respectively).

How Do I Configure It?

Here is an example of how to set up Linux to automatically mount an s3backer filesystem using NBD at startup.

First, create a file with your favorite s3backer options. Do not include the mount point and bucket name; This allows you to use this file in either FUSE or NBD mode.

# /home/archie/etc/s3backer-options.conf
# s3backer options

# Sizing
--size=1t
--blockSize=256k

# Credentials
--accessFile=/home/archie/.s3b_passwd

# List blocks on startup
--listBlocks
--listBlocksThreads=50

# Encryption
--ssl
--encrypt
--passwordFile=/home/archie/.s3b_key

# Block cache
--blockCacheSize=20000
--blockCacheFile=/home/archie/.s3b_cache
--blockCacheWriteDelay=15000
--blockCacheThreads=4
--blockCacheRecoverDirtyBlocks
--blockCacheNumProtected=1000

# Misc
--timeout=90

Next, install a systemd(1) service file so that s3backer is started automatically as soon as the network is ready. The configuration below will use the above options file and serve the virtual disk in S3 bucket mybucket on device /dev/nbd0:

# /usr/lib/systemd/system/s3backer-nbd.service
# systemd service file for running s3backer in NBD mode

[Unit]
Description=s3backer running in NBD mode
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/archiecobbs/s3backer

[Install]
WantedBy=multi-user.target

[Service]
Type=forking
ExecStart=/usr/bin/s3backer --nbd --configFile=/home/archie/etc/s3backer-options.conf mybucket /dev/nbd0

# Security hardening
ProtectSystem=full
#ProtectHome=read-only
ProtectHostname=true
ProtectClock=true
ProtectKernelTunables=true
ProtectKernelLogs=true
ProtectControlGroups=true
RestrictRealtime=true

To make that service file become active, do this:

$ systemctl daemon-reload
$ systemctl enable s3backer-nbd.service

Finally, add an entry to /etc/fstab to mount the disk wherever you want it mounted:

# /etc/fstab

# s3backer using NBD
/dev/nbd0  /opt/s3b   ext4   _netdev,discard,noatime,nodiratime,x-systemd.requires=s3backer-nbd.service 0 0

Now on your next boot everything should start up automatically:

$ systemctl status s3backer-nbd.service 
● s3backer-nbd.service - s3backer running in NBD mode
     Loaded: loaded (/usr/lib/systemd/system/s3backer-nbd.service; enabled; vendor preset: disabled)
     Active: active (running) since Sun 2022-07-31 15:45:49 CDT; 8min ago
       Docs: https://github.com/archiecobbs/s3backer
    Process: 1398 ExecStart=/usr/bin/s3backer --nbd --configFile=/home/archie/etc/s3backer-options.conf mybucket /dev/nbd0 (code=exited, status=0/SUCCESS)
      Tasks: 23 (limit: 2342)
     CGroup: /system.slice/s3backer-nbd.service
             ├─1503 /usr/sbin/nbdkit --filter=exitlast --unix /var/run/s3backer-nbd/0000000000000006_00000000000044a0 s3backer s3b_configFile=/home/archie/etc/s3backer-options.conf bucket=mybucket
             └─1504 /usr/bin/s3backer --nbd --configFile /home/archie/etc/s3backer-options.conf mybucket /dev/nbd0

Jul 31 15:45:48 rebound s3backer[1438]: nbdkit: auto-detecting block size and total file size...
Jul 31 15:45:49 rebound s3backer[1438]: nbdkit: auto-detected block size=256k and total size=1t
Jul 31 15:45:49 rebound s3backer[1438]: 2022-07-31 15:45:49 INFO: reading meta-data from cache file `/home/archie/.s3b_cache'
Jul 31 15:45:49 rebound s3backer[1438]: 2022-07-31 15:45:49 INFO: loaded cache file `/home/archie/.s3b_cache' with 0 free and 20000 used blocks (max index 20000)
Jul 31 15:45:49 rebound s3backer[1438]: 2022-07-31 15:45:49 INFO: established new mount token 0x55688ede
Jul 31 15:45:49 rebound s3backer[1398]: s3backer: connecting mybucket to /dev/nbd0 and daemonizing
Jul 31 15:45:49 rebound systemd[1]: Started s3backer running in NBD mode.
Jul 31 15:45:49 rebound nbdkit[1503]: s3backer: starting non-zero block survey
Jul 31 15:45:49 rebound nbdkit[1503]: s3backer: mounting mybucket
Jul 31 15:45:56 rebound nbdkit[1503]: s3backer: non-zero block survey completed (58417 non-zero blocks reported)

$ df
Filesystem      1K-blocks     Used Available Use% Mounted on
...
/dev/nbd0      1056763060  9692540 993367048   1% /opt/s3b