v5.0.0: backup/restore apps, overhaul TTS, add new node to existing cluster + 🐛 fixes

[jessebot]

Features

Sensitive Values Overhaul

• You can now specify your own sensitive values via environment variables using the new value_from map for both apps.APP_NAME.init.values and any value under apps.APP_NAME.backup
  • also has initial support for bitwarden though it's still untested, and openbao is also coming
  • the apps.APP_NAME.init.sensitive_values list has been removed
  • example of providing a sensitive value for init values:

apps:
  nextcloud:
    init:
      values:
        admin_user: nextcloud_admin_user
        smtp_user: my-smtp-username
        smtp_host: my-smtp-host.com
        # this value is taken from an external source
        smtp_password:
          value_from:
            # you can change this to any environment variable present at the time of running smol-k8s-lab
            env: NC_SMTP_PASSWORD

NOTE: the TUI doesn't support setting sensitive values via value_from right now. It will just pull your sensitive value and change it to dots. This feature will come at a later date.

Backups and Restores

_{currently only supported for a handful of apps (nextcloud, matrix, mastodon, home assistant, and zitadel), but more coming soon!}

• Added support for seaweedfs/app specific pvc restores, and postgresql restores.
• Added support for specifying backup times (in cron syntax)
• Also added support for Nextcloud maintenance window time (cron syntax for both start and end time)
• We now support a global StorageClassName called apps_global_config.pvc_storage_class in the yaml (used by default for nextcloud and matrix right now)
• Each app that supports backups and restores now has a tab for both on the apps screen and a section in the config.yaml.
• The backup config.yaml section looks like this:

apps:
  nextcloud:
    backups:
      # cronjob syntax schedule to run nextcloud pvc backups
      pvc_schedule: 10 0 * * *
      # cronjob syntax (with SECONDS field) for nextcloud postgres backups
      # must happen at least 10 minutes before pvc backups, to avoid corruption
      # due to missing files. This is because the cnpg backup shows as completed
      # before it actually is, due to the wal archive it lists as it's end not
      # being in the backup yet
      postgres_schedule: 0 0 0 * * *
      s3:
        # these are for pushing remote backups of your local s3 storage, for speed and cost optimization
        endpoint: s3.eu-central-003.backblazeb2.com
        bucket: my-nextcloud-bucket
        region: eu-central-003
        secret_access_key:
          value_from:
            env: NC_S3_BACKUP_SECRETKEY
        access_key_id:
          value_from:
            env: NC_S3_BACKUP_ACCESS_ID
      restic_repo_password:
        value_from:
          env: NC_RESTIC_REPO_PASS

• The restore section looks like this:

apps:
  nextcloud:
    init:
      enabled: true
      # this is the restore section, as it's a type of initialization
      restore:
        enabled: true
        # for postgresql cluster restores using the cloud native postgresql operator
        cnpg_restore: true
        # these can all be set to any restic snapshot ID (long or short), but they default to latest
        restic_snapshot_ids:
          seaweedfs_volume: latest
          seaweedfs_filer: latest
          seaweedfs_master: latest
          nextcloud_files: latest

Overhaul the text to speech features to be their own widget called SmolAudio

• pre-generated audio files for each common thing that would need to be said, but you can still use your own TTS CLI if you'd like by providing smol_k8s_lab.tui.accessibility.text_to_speech.speech_program
• sets a default language of "en" for english, but dutch (nl) is also partially available in the config.yml
  • creates a config/audio/en.yaml and config/audio/nl.yaml for custom titles, descriptions, phrases, and common words
• you can now enable and disable screen titles and/or descriptions separately
• creates a small dev-only program called smol-tts for generating text to speech audio files for each language
• tab names for tabbed content, switch values, and drop down menu values are now read out for when pressing F5
• we've re-created all the screen descriptions and and separated out screen titles from screen descriptions

New nodes for k3s after cluster is already up

• You can now specify an SSH port when adding new nodes to k3s installs (defaults to 22)
• Adds a "modify nodes" button to the cluster modal screen for k3s, so you can add nodes after the cluster is already installed on a new screen

Release process

We will now attempt to release an appimage each time we release :) This will assist in ensuring the brew install goes more smoothly in the future. This has never been done for this project before, so we expect some initial growing pains on this. Please be patient as we get a consistent appimage.

Misc

• we now support python 3.12!
• clusters datatable now displays the OS platform and the kubernetes version for each row
• in the TUI, there are now sync/delete links baked into the bottom border for each app that is available in ArgoCD and enabled. Closes Delete app via the TUI #109
• TUI config screen's k9s section is removed entirely and the whole accessibility section is now at the top of the screen
• k9s section of both the TUI and the CLI has been replaced with run_command section that allows you to run commands either during smol-k8s-lab's app config phase, or after it. It looks like this:

smol_k8s_lab:
  run_command:
    # command to run after smol-k8s-lab tui is done or immediately when running
    command: k9s --command applications.argoproj.io
    # tell me which terminal you use if you'd like to use split or tab features, only supports wezterm and zellij right now. submit issue/PR for more options :)
    terminal: wezterm
    # where to run the command, options: same window, new window, new tab, split left, split right, split top, split bottom
    # if set to "same window", we just run the command in the same window after we're done the entire smol-k8s-lab cli run
    window_behavior: split right

• you can now specify a destination cluster for your Argo CD App. Closes Feature Request: Allow specification of ArgoCD App destination server in smol-k8s-lab config.yaml #211 Example:

apps:
  zitadel:
    argo:
      # git repo to install the Argo CD app from
      repo: https://github.com/small-hack/argocd-apps
      # path in the argo repo to point to. Trailing slash very important!
      path: zitadel/app_of_apps/
      # either the branch or tag to point at in the argo repo above
      revision: add-pvc-helm-chart-for-nextcloud
      # kubernetes cluster to install the k8s app into, defaults to Argo CD default
      cluster: https://kubernetes.default.svc
      # namespace on destination cluster to install the k8s app in
      namespace: zitadel
      # recurse directories in the provided git repo
      directory_recursion: false

Bug Fixes

• CNPG operator is now installed during the operator phase of installs
• print element web interface URL instead of matrix URL at the end of the run
• fix defaults for Bitwarden CLI env vars: BW_HOST, BW_SESSION
• fix bug where when completely unauthenticated to bitwarden, the unlock never happened. now if not authenticated, we authenticate, then unlock the vault.
• fix bug where if apps_global_config.external_secrets was set to bitwarden, the bitwarden credentials were requested even if password manager was disabled AND the external secrets operator app was disabled
• generic device plugin could crash if it didn't have enough memory so we switched to helm chart for the same app
• changes button at bottom of new nodes box to say "➕ node" for "add remote nodes" tab
• new node and new option key bindings were previously not accessible if you were already focused on an input field. to solve this, we've changed that key binding to be ctrl+n so it's always something that can be pressed
• changes the cancel button the cluster modal to be in the bottom border of the screen to preserve screen real estate
• fixes padding and gutters for the css of the add nodes tab and screen for k3s
• fixed issue where when screen titles were enabled, they were read before the widgets on the screen were fully loaded

Misc changes

• Cleaned up a bunch of whitespace
• add some more comments to the default config.yaml
• updates default input_field function to include a default empty validator
• adds name of cluster being edited to the title of most screens
• did a cleanup pass of the docs website

outstanding tasks

• document backups with nextcloud and matrix
• document restore process with nextcloud and matrix
• add contributing page and planned/requested features page to docs
• update generic device plugin docs
• verify/create bitwarden docs

This PR will be merged in conjuction with: small-hack/argocd-apps#695

[jessebot]

just some docs to do, and then we're all set :D