[BUG] Bastille export is broken in 0.9.20210714

[hackacad]

[MANDATORY] Describe the bug [MANDATORY]
bastille export jail seems to be broken in the latest release

root@bastilledev:~ # bastille export jail
Error: Stream can not be written to a terminal.
You must redirect standard output.

[MANDATORY] Bastille and FreeBSD version (paste bastille -v && freebsd-version -kru output)
0.9.20210714
13.0-RELEASE-p3
13.0-RELEASE-p3
13.0-RELEASE-p3

[MANDATORY] How did you install bastille? (port/pkg/git)
pkg/latest
github/master
[optional] Steps to reproduce?
bastille export jail

[optional] Additional context
workaround:
add --tgz

[JRGTH]

Hi Sven, its not a bug, the user just now need to specify the wanted predefined export compress/format, otherwise you need to redirect the stream.

Export usage:

# bastille export
Usage: bastille export | option(s) | TARGET | PATH
    Options:

         --gz       -- Export a ZFS jail using GZIP(.gz) compressed image.
    -r | --raw      -- Export a ZFS jail to an uncompressed RAW image.
    -s | --safe     -- Safely stop and start a ZFS jail before the exporting process.
         --tgz      -- Export a jail using simple .tgz compressed archive instead.
         --txz      -- Export a jail using simple .txz compressed archive instead.
    -v | --verbose  -- Be more verbose during the ZFS send operation.
         --xz       -- Export a ZFS jail using XZ(.xz) compressed image.

Tip: If no option specified, container should be exported to standard output.

Here in this link are more information about export/import redirection.

[hackacad]

As this is a breaking change it should at least be documented in the release notes.

Imho there should still be a default format (.tgz as before). @cedwards ?

[g-teley]

I can add to that that I experience other issues.
When doing an export from a jail and sending it to another system for import, I end up with a copy of a different jail!
So:
Export Jail A > backup_of_jail_A.gz
Copy to other system.
Import backup_of_jail_A.gz
Console into 'new' jail.
Experience you ended up in a copy of Jail-B!

What I can see in the export-file is that it starts with the correct fstab but in the jail.conf file references the wrong ZFS path.
What's particular weird is that the IP addres is taken from the required jail but the rest (data) is from the wrong jail.
More over, the name of the ZFS snapshot references the required jail, but as said, it containes the data from the wrong jail.

[JRGTH]

Exporting a jail will retain/references to its current config, so importing to another system may requires some manual editing if an existing jail is already using similar settings, for example IP, paths etc, however ZFS path/mount-point should be configured upon import, and regarding fstab this should be edited regarding destination to avoid mounting issues.

However your issue is quite strange and I will ask for more information in this regards about host/jail/filesystem/paths, target import/filesystem/paths etc.

Between ending with a wrong jail, said exporting jail-A then it wrongly imports the jail-B seems more like a jails-similar-naming and/or grep/regex issue or alike, and I would like to reproduce this regardless.

Also does this only happens with .gz image export only?, have you tried other exports formats as well.

Regards

[g-teley]

Sorry for the delay but attached you'll find a transcript to reproduce the error.
Bastille export-import problem.pdf

[g-teley]

One more addition: today I noticed that doing a bastille export -s jail_name resulted in a ZFS snapshot with a date in its name that references a date in the past. I assume this is the date at which I created another jail of which the new ones are clones.
After cloning, I cleaned them up and installed the programs/services I needed.
Apparently this works great, until you want to transfer them to another system with export/import.

[JRGTH]

One more addition: today I noticed that doing a bastille export -s jail_name resulted in a ZFS snapshot with a date in its name that references a date in the past. I assume this is the date at which I created another jail of which the new ones are clones. After cloning, I cleaned them up and installed the programs/services I needed. Apparently this works great, until you want to transfer them to another system with export/import.

Hello I got your PDF file and I will try this issue asap due holy days.

Regarding to "transfer" a jail to another system, be aware that the predefined export/import options is to export the jail on the current system for backup purposes, however you can import them to another system by copying the export files to the other system then import the jail there.

However in order to transfer to another system say a remote machine, you need to redirect(advanced) the jail which is way more complex than the predefined local export/import options, also the user may need extensive experience with ssh, zfs and pipe the wanted custom commands, IIRC I have some basic explanation and examples about the jail export/import I/O redirection somewhere in the commits comments but I have to dig through to post the link.

For jails export redirection to standard output just use: bastille export jailname > custom_args_here

# bastille export
Usage: bastille export | option(s) | TARGET | PATH
    Options:

         --gz       -- Export a ZFS jail using GZIP(.gz) compressed image.
    -r | --raw      -- Export a ZFS jail to an uncompressed RAW image.
    -s | --safe     -- Safely stop and start a ZFS jail before the exporting process.
         --tgz      -- Export a jail using simple .tgz compressed archive instead.
         --txz      -- Export a jail using simple .txz compressed archive instead.
    -v | --verbose  -- Be more verbose during the ZFS send operation.
         --xz       -- Export a ZFS jail using XZ(.xz) compressed image.

Tip: If no option specified, container should be exported to standard output.

Edit: I found the export/import I/O commit with the comments in this link

Additionally for bastille/zfs users, zfs send/recv can be used to fully replicate a jail to another system(advanced), however this is a very broad topic and requires some zfs knowledge and manual configuration edits.

[g-teley]

Hello, After some investigations I'm confident that the problem is caused by `bastille export` taking the wrong ZFS snapshot. The clarify how this has happened: I started with a clean FreeBSD 13.0 server, installed Bastille and created the first jail by importing an Ezjail archive from a jail called 'Support'. When this proved to be working fine, I cloned this jail to two different jails 'Koobo_Web' and 'Koobo_Mail'. After cleaning these jails and filling them with backups of their respected (physical) servers I tested them on correct functionality. This resulted in three jails on my test server 'Support', 'Koobo_Web' and 'Koobo_Mail'. All working as intended. Thereafter I created a VirtualBox FreeBSD system and installed Bastille. (The VirtualBox is intended as the final backup for a customer) I exported both 'Koobo_Mail' and "Koobo_web' and copied them (scp) to the VirtualBox system. After importing them I noticed that 'Koobo_Web' worked as expected, but that 'Koobo_Mail' actually was a copy op 'Support'. All naming and references in the jail.conf for "Koobo_Mail" are correct. But as said before, the content is from the original 'Support' jail. What I don't understand is why an export of the 'Koobo_Web' jail worked fine, but the oter didn't. Hth. Op wo 8 dec. 2021 om 22:03 schreef JRGTH ***@***.***>:

…

One more addition: today I noticed that doing a bastille export -s jail_name resulted in a ZFS snapshot with a date in its name that references a date in the past. I assume this is the date at which I created another jail of which the new ones are clones. After cloning, I cleaned them up and installed the programs/services I needed. Apparently this works great, until you want to transfer them to another system with export/import. Hello I got your PDF file and I will try this issue asap. Regarding to "transfer" a jail to another system, be aware that the predefined export/import options is to export the jail on the current system for backup purposes, however you can import them to another system by copying the export files to the other system then import the jail there. However in order to transfer to another system say a remote machine, you need to redirect(advanced) the jail which is way more complex than the predefined local export/import options, also the user may need extensive experience with ssh, zfs and pipe the wanted custom commands, IIRC I have some basic explanation and examples about the jail export/import I/O redirection somewhere in the commits comments but I have to dig through to post the link. For jails export redirection to standard output just use: bastille export jailname > custom_args_here # bastille export Usage: bastille export | option(s) | TARGET | PATH Options: --gz -- Export a ZFS jail using GZIP(.gz) compressed image. -r | --raw -- Export a ZFS jail to an uncompressed RAW image. -s | --safe -- Safely stop and start a ZFS jail before the exporting process. --tgz -- Export a jail using simple .tgz compressed archive instead. --txz -- Export a jail using simple .txz compressed archive instead. -v | --verbose -- Be more verbose during the ZFS send operation. --xz -- Export a ZFS jail using XZ(.xz) compressed image. Tip: If no option specified, container should be exported to standard output. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#443 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABDOXUBVJR5VR5ASJNXHRHLUP7BZRANCNFSM5DWQEVAA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

-- Met vriendelijke groet, Guus Teley EMITA CISA RI Heythuysen

[cedwards]

I would like export to use a sane option by default.

Ran into this issue myself and wasn't immediately clear why the export wasn't working. I think --tgz is a sane default.

[JRGTH]

I still cannot reproduce the above export error denoted by g-teley wen debugging my own server.

root@nas-mserver: ~# ls /mnt/storage/bastille/jails/jail_test
File_1    File_2    fstab     jail.conf root
root@nas-mserver: ~# bastille clone jail_test jail_cloned 10.0.0.11
Valid: (10.0.0.11).
Attempting to clone 'jail_test' to jail_cloned...
Cloned 'jail_test' to 'jail_cloned' successfully.
root@nas-mserver: ~# ls /mnt/storage/bastille/jails/jail_cloned
File_1    File_2    fstab     jail.conf root
root@nas-mserver: ~# touch /mnt/storage/bastille/jails/jail_cloned/New_File_3
root@nas-mserver: ~# touch /mnt/storage/bastille/jails/jail_cloned/New_File_4
root@nas-mserver: ~# ls /mnt/storage/bastille/jails/jail_cloned
File_1     File_2     New_File_3 New_File_4 fstab      jail.conf  root
root@nas-mserver: ~# bastille export -v --gz jail_cloned 
Exporting 'jail_cloned' to a compressed .gz image...
Creating temporary ZFS snapshot for export...
Sending ZFS data stream...
full send of storage/bastille/jails/jail_cloned@bastille_jail_cloned_2021-12-27-070352 estimated size is 54.6K
full send of storage/bastille/jails/jail_cloned/root@bastille_jail_cloned_2021-12-27-070352 estimated size is 6.99M
total estimated size is 7.04M
full send of storage/bastille/jails/jail_cloned@bastille_jail_cloned_2021-12-27-070352 estimated size is 54.6K
TIME        SENT   SNAPSHOT
full send of storage/bastille/jails/jail_cloned/root@bastille_jail_cloned_2021-12-27-070352 estimated size is 6.99M
TIME        SENT   SNAPSHOT
 85.8%
Exported '/mnt/storage/bastille/backups/jail_cloned_2021-12-27-070352.gz' successfully.
root@nas-mserver: ~# bastille destroy jail_cloned
Deleting Jail: jail_cloned.

root@nas-mserver: ~# bastille import jail_cloned_2021-12-27-070352.gz
Validating file: jail_cloned_2021-12-27-070352.gz...
File validation successful!
Importing 'jail_cloned' from compressed .gz image.
Receiving ZFS data stream...
/mnt/storage/bastille/backups/jail_cloned_2021-12-27-070352.gz:	   85.8%
Container 'jail_cloned' imported successfully.
root@nas-mserver: ~# ls /mnt/storage/bastille/jails/jail_cloned
File_1     File_2     New_File_3 New_File_4 fstab      jail.conf  root

Still doing some more investigations on VM's before testing back on my hardware.

[JRGTH]

I would like export to use a sane option by default.

Ran into this issue myself and wasn't immediately clear why the export wasn't working. I think --tgz is a sane default.

Hi cedwards, the user now must specify a built-in export compression method, if no user option is specified, the export stream must be redirected to standard output, and this is very similar when exporting an Boot Environment like either bectl or beadm*

Example export with no user option:

# bastille export jellyfin
Error: Stream can not be written to a terminal.
You must redirect standard output.

# bectl export zroot
bectl export: must redirect output

Bastille Export Usage:

# bastille export
Usage: bastille export | option(s) | TARGET | PATH
    Options:

         --gz       -- Export a ZFS jail using GZIP(.gz) compressed image.
    -r | --raw      -- Export a ZFS jail to an uncompressed RAW image.
    -s | --safe     -- Safely stop and start a ZFS jail before the exporting process.
         --tgz      -- Export a jail using simple .tgz compressed archive instead.
         --txz      -- Export a jail using simple .txz compressed archive instead.
    -v | --verbose  -- Be more verbose during the ZFS send operation.
         --xz       -- Export a ZFS jail using XZ(.xz) compressed image.

Tip: If no option specified, container should be exported to standard output.

Example on bastille export redirect:(I've used -v|--verbose optionally)

# bastille export --verbose jellyfin > /tmp/jellyfin_backup
full send of storage/bastille/jails/jellyfin@bastille_jellyfin_2021-12-27-072430 estimated size is 54.6K
full send of storage/bastille/jails/jellyfin/root@bastille_jellyfin_2021-12-27-072430 estimated size is 3.11G
total estimated size is 3.11G
full send of storage/bastille/jails/jellyfin@bastille_jellyfin_2021-12-27-072430 estimated size is 54.6K
TIME        SENT   SNAPSHOT
full send of storage/bastille/jails/jellyfin/root@bastille_jellyfin_2021-12-27-072430 estimated size is 3.11G
TIME        SENT   SNAPSHOT
07:24:51   3.15G   storage/bastille/jails/jellyfin/root@bastille_jellyfin_2021-12-27-072430

*Setting a defined export option will break the more advanced and flexible user export redirection.

Alternatively I can make the export command to always use a predefined user export/backup option by either reading ${bastille_export_options} from the bastille config file or simply by setting an environment variable like below:

export bastille_export_options=--tgz

What do you think?
Regards

[hackacad]

Adding a Bastille.conf parameter sounds reasonable as well. Second choice would be showing an error message that no compression method has be selected.

[g-teley]

I would opt for both. Requiring a compression parameter value and erroring out when none given. Met vriendelijke groet, Guus Teley (mobiel) Op di 4 jan. 2022 07:20 schreef Sven R ***@***.***>:

…

Adding a Bastille.conf parameter sounds reasonable as well. Second choice would be showing an error message that no compression method has be selected. — Reply to this email directly, view it on GitHub <#443 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABDOXUBO35TARIKIYOADZETUUKGRHANCNFSM5DWQEVAA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>

[JRGTH]

Sounds reasonable, since no export option defined means that the user will manually redirect/pipe the ZFS stream to either local or remote location, I will make the export command to override the options if the ${bastille_export_options} variable is set.

Additionally I will add more explanatory information to the error output, as manual redirection is avail only on ZFS setups as well.

Regards

[JRGTH]

This issue should be fixed in this commits: 26e8f38

Regards

[cedwards]

@hackacad fixes for this were merged in #479 . Can you please tell us if this resolves your issue?

[g-teley]

I've exported again the jail where I noticed the errors. This time there were no references to the ZFS dataset where the original jail came from, so it looks good.

[hackacad]

Great job, thanks @JRGTH and @cedwards

[JRGTH]

I've exported again the jail where I noticed the errors. This time there were no references to the ZFS dataset where the original jail came from, so it looks good.

Hello, thanks for reporting this is working for you as expected now, the taken snapshots for export now has the jail name in question for ease reference/debug bastille_${TARGET}_${DATE}