secrets

Secrets Management

For Website/App's passwords, see /home/base/desktop/password-store for more details.

All my secrets are safely encrypted via agenix, and stored in a separate private GitHub repository and referenced as a flake input in this flake.

The encryption is done using the public keys of all my hosts (/etc/ssh/ssh_host_ed25519_key), so that they can only be decrypted on any of my configured hosts. The host keys are generated locally on each host by OpenSSH without a passphrase and are only readable by root. The host keys will never leave the host.

In this way, all secrets are still encrypted when transmitted over the network and written to /nix/store. They are decrypted only when they are finally used.

In addition, we further improve the security of secret files by storing them in a separate private repository.

This directory contains this README.md, and a nixos.nix/darwin.nix file that is used to decrypt all my secrets via agenix. Then, I can use them in this flake.

Adding or Updating Secrets

All the operations in this section should be performed in my private repository: nix-secrets.

This task is accomplished using the agenix CLI tool with the ./secrets.nix file, so you need to have it installed first:

To use agenix temporarily, run:

nix shell github:ryantm/agenix#agenix

or agenix provided by ragenix, run:

nix shell github:ryan4yin/ragenix#ragenix

Suppose you want to add a new secret file xxx.age. Follow these steps:

1. Navigate to your private nix-secrets repository.
2. Edit secrets.nix and add a new entry for xxx.age, defining the encryption keys and the secret file path, for example:

# This file is not imported into your NixOS configuration. It is only used for the agenix CLI.
# agenix use the public keys defined in this file to encrypt the secrets.
# and users can decrypt the secrets by any of the corresponding private keys.

let
  # Get system's ssh public key by command:
  #    cat /etc/ssh/ssh_host_ed25519_key.pub
  # If you do not have this file, you can generate all the host keys by command:
  #    sudo ssh-keygen -A
  idol_ai = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINHZtzeaQyXwuRMLzoOAuTu8P9bu5yc5MBwo5LI3iWBV root@ai";
  harmonica = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINT7Pgy/Yl+t6UkHp5+8zfeyJqeJ8EndyR1Vjf/XBe5f root@harmonica";
  fern = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMokXUYcUy7tysH4tRR6pevFjyOP4cXMjpBSgBZggm9X root@fern";

  # A key for recovery purpose, generated by `ssh-keygen -t ed25519 -a 256 -C "ryan@agenix-recovery"` with a strong passphrase
  # and keeped it offline in a safe place.
  recovery_key = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHnIGH+653Oe+GQaA8zjjj7HWMWp7bWXed4q5KqY4nqG ryan@agenix-recovery";
  systems = [
    idol_ai
    harmonica
    fern

    recovery_key
  ];
{
  "./xxx.age".publicKeys = users ++ systems;
}

3. Create and edit the secret file xxx.age interactively using the following command:

sudo agenix -e ./xxx.age -i /etc/ssh/ssh_host_ed25519_key

Alternatively, you can encrypt an existing file to xxx.age using the following command:

cat xxx | sudo agenix  -e ./xxx.age -i /etc/ssh/ssh_host_ed25519_key

agenix will encrypt the file with all the public keys we defined in secrets.nix, so all the users and systems defined in secrets.nix can decrypt it with their private keys.

Deploying Secrets

All the operations in this section should be performed in this repository.

First, add your own private nix-secrets repository and agenix as a flake input, and pass them to sub modules via specialArgs:

{
  inputs = {
    # ......

    # secrets management, lock with git commit at 2023/5/15
    agenix.url = "github:ryantm/agenix/db5637d10f797bb251b94ef9040b237f4702cde3";

    # my private secrets, it's a private repository, you need to replace it with your own.
    mysecrets = { url = "github:ryan4yin/nix-secrets"; flake = false; };
  };

  outputs = inputs@{ self, nixpkgs, ... }: {
    nixosConfigurations = {
      nixos-test = nixpkgs.lib.nixosSystem {
        system = "x86_64-linux";

        # Set all input parameters as specialArgs of all sub-modules
        # so that we can use the `agenix` & `mysecrets` in sub-modules
        specialArgs = inputs;
        modules = [
          # ......

          # import & decrypt secrets in `mysecrets` in this module
          ./secrets/default.nix
        ];
      };
    };
  };
}

Then, create ./secrets/default.nix with the following content:

# import & decrypt secrets in `mysecrets` in this module
{ config, pkgs, agenix, mysecrets, ... }:

{
  imports = [
     agenix.nixosModules.default
  ];

  # if you changed this key, you need to regenerate all encrypt files from the decrypt contents!
  age.identityPaths = [
    # using the host key for decryption
    # the host key is generated on every host locally by openssh, and will never leave the host.
    "/etc/ssh/ssh_host_ed25519_key"
  ];

  age.secrets."xxx" = {
    # whether secrets are symlinked to age.secrets.<name>.path
    symlink = true;
    # target path for decrypted file
    path = "/etc/xxx/";
    # encrypted file path
    file =  "${mysecrets}/xxx.age";  # refer to ./xxx.age located in `mysecrets` repo
    mode = "0400";
    owner = "root";
    group = "root";
  };
}

From now on, every time you run nixos-rebuild switch, it will decrypt the secrets using the private keys defined in age.identityPaths. It will then symlink the secrets to the path defined by the age.secrets.<name>.path argument, which defaults to /etc/secrets.

Adding a new host

1. cat the system-level public key(/etc/ssh/ssh_host_ed25519_key.pub) of the new host, and send it to an old host which has already been configured.
2. On the old host:
   1. Add the public key to secrets.nix, and rekey all the secrets via sudo agenix -r -i /etc/ssh/ssh_host_ed25519_key.
   2. Commit and push the changes to nix-secrets.
3. On the new host:
   1. Clone this repo and run nixos-rebuild switch to deploy it, all the secrets will be decrypted automatically via the host private key.

Troubleshooting

1. Nix-Darwin Module

Check logs:

tail -n 100 /Library/Logs/org.nixos.activate-agenix.stderr.log
tail -n 100 /Library/Logs/org.nixos.activate-agenix.stdout.log

2. NixOS Module

Check logs:

journalctl | grep -5 agenix

Other Replacements

• ragenix: A Rust reimplementation of agenix.
  • agenix is mainly written in bash, and it's error message is quite obscure, a little typo may cause some errors no one can understand.
  • with a type-safe language like Rust, we can get a better error message and less bugs.