modules/age: symlink files into place #27

cole-h · 2021-02-25T10:31:24Z

This follows sops-nix's implementation, where it creates a
/run/secrets.d ramfs mountpoint and a "generation" each time
the activation script runs, and then symlinks /run/secrets to
/run/secrets.d/[generation].

Fixes #26.
Fixes #21.

~~TODO (maybe): is it possible to not create a new generation each time activation runs (if nothing changed)? Does sops-nix do this?~~ While possible, this would introduce a decent amount of complexity for (IMHO) no gain. Since we remove old secrets generations, I think this is unnecessary.

modules/age.nix

cole-h · 2021-02-27T04:38:09Z

Old secrets generations are removed in 368584b. From what I can tell with my last few minutes of testing, works well. ls -al /run/secrets.d shows only the latest generation.

modules/age.nix

cole-h · 2021-03-02T09:42:38Z

Been using this for a few days and it's been working flawlessly. Switching secrets, rebooting, the whole kit and caboodle.

modules/age.nix

asymmetric

In general, I find this a bit harder to reason about than the previous solution.

For example: why do we need /run/secrets? Can't we symlink /run/secrets.d/N into secretType.path?

modules/age.nix

asymmetric · 2021-03-23T09:08:35Z

modules/age.nix

    (umask 0400; LANG=${config.i18n.defaultLocale} ${ageBin} --decrypt ${identities} -o "$TMP_FILE" "${secretType.file}")
    chmod ${secretType.mode} "$TMP_FILE"
    chown ${secretType.owner}:${secretType.group} "$TMP_FILE"
-    mv -f "$TMP_FILE" '${secretType.path}'
+    mv -f "$TMP_FILE" "${cfg.secretsMountPoint}/$_count/${secretType.name}"
+    [ "${secretType.path}" != "/run/secrets/${secretType.name}" ] && ln -sfn "/run/secrets/${secretType.name}" "${secretType.path}"


So IIUC, if secretType.path is e.g. /foo/bar, the secret will be symlinked both into /foo/bar and /run/secrets/bar.

Is this necessarily what a user wants?

I'd say yes (and it's how sops-nix behaves, IIRC). /run/secrets should be the single source of truth for the secrets on the system, so that switching a secrets generation "expires" any removed secrets (resulting in a dead symlink, if names don't collide). (While /run/secrets.d/## is the actual source of truth, /run/secrets is a static reference to it; otherwise, it would be hard to refer to secrets when they change with every switch of the configuration.)

This would definitely increase complexity, but maybe we could store a sort of manifest in /run/secrets.d that lists the secret name and its secretType.path, so we can unlink them all prior to switching secrets generations.

modules/age.nix

kclejeune · 2021-05-09T19:42:50Z

Hey @ryantm, sorry to bump this, but just wanted to see if there's anything blocking this from being merged? This project seems really nice but this is the one feature holding me back from using it over sops-nix.

cole-h · 2021-05-10T18:24:30Z

Rebased and squashed.

modules/age.nix

alarsyo · 2021-09-18T22:17:41Z

Is there anything missing to get this merged? 😅

kclejeune · 2021-10-20T21:42:12Z

Is there anything missing to get this merged? 😅

Bump? 🙃

cole-h · 2021-10-20T21:52:32Z

@ryantm Are you only waiting on me to document the change of /run/secrets to /run/agenix, or is there more to be done? I'm satisfied with the implementation, but am happy to make any changes you deem necessary.

(FWIW, I've been running against this branch for the past few months; no troubles.)

ryantm · 2021-10-20T21:53:12Z

This project seems really nice but this is the one feature holding me back from using it over sops-nix.

Maybe I knew why at some point, but I don't remember now. Is there some good explanation somewhere for why you want to change it to work like this?

kclejeune · 2021-10-20T22:14:43Z

This project seems really nice but this is the one feature holding me back from using it over sops-nix.

Maybe I knew why at some point, but I don't remember now. Is there some good explanation somewhere for why you want to change it to work like this?

I'd like to be able to use this for system-level secret management, not just for secrets in my nix configuration. Symlinking to /run/secrets allows me to access them in a consistent place.

ryantm · 2021-10-20T23:05:50Z

@kclejeune Thanks for explaining. I think it sounds like you are saying you have a NixOS machine that has some program that is not managed by NixOS that you want to be able to access a secret that is managed by NixOS. Is that right?

kclejeune · 2021-10-20T23:11:12Z

@kclejeune Thanks for explaining. I think it sounds like you are saying you have a NixOS machine that has some program that is not managed by NixOS that you want to be able to access a secret that is managed by NixOS. Is that right?

That's correct! My primary use case is actually on nix-darwin, where I can't manage as many things declaratively, but the same need exists on my NixOS system too.

Gerschtli · 2021-11-08T13:15:27Z

It would be really helpful if one could specify if the secret to be installed should be created as a symlink or just a copy because some tools like openssh do not allow e.g. ssh keys to be symlinks.

ryantm · 2021-11-08T13:26:21Z

I've also noticed this problem with symlinked secrets in the Java ecosystem. Specifically Elasticsearch.

Gerschtli · 2021-11-08T13:59:15Z

The main question would be, how copied secrets will be cleaned up if they are removed from the nixos configuration.. Otherwise this could lead to non reproducible system when the system relies on secret file that are no more present in the configuration.

cole-h · 2021-11-08T15:56:39Z

Unless I'm mistaken, I don't think the current implementation in master takes care of cleaning up files anyways, so... That can probably be a separate issue / PR.

If all that's necessary to make this mergeable is:

Update the docs
Add a per-secret option to symlink or copy secrets (defaulting to symlink)

I can do that.

ryantm · 2021-11-08T16:00:02Z

@cole-h yeah I think that's it (aside from current merge conflicts) thank you!

cole-h · 2021-11-08T17:52:28Z

@ryantm Is there any place in particular you'd like me to call out the switch from /run/secrets to /run/agenix? That's all that's left (hopefully :D)!

ryantm · 2021-11-10T15:47:49Z

@cole-h I think we can just put something additional in the release notes.

Is it the case that we are relying on $_count to not be clobbered by some other activation script? I'm thinking it should be named something else if that is case, like $agenix_generation. I was wondering if it is possible to use the same generation number as NixOS. Then we wouldn't need to generate a variable and it might be slightly helpful for debugging things.

cole-h · 2021-11-16T03:26:24Z

Sorry, this slipped my mind! I'll get back to it this weekend.

I'll make the change to $_agenix_generation, but I'm not enthused by the idea of tying it to the NixOS generation -- mostly because that'll still be complicated.

How do we get the generation? Reading the /nix/var/nix/profiles/system symlink isn't robust enough because it's possible to install the system under a different profile name (see nixos-rebuild(8) and search for --profile-name). What if this is on brand a new install? I doubt there's a generation to read / parse at that point.

kclejeune · 2021-11-16T03:37:52Z

Sorry, this slipped my mind! I'll get back to it this weekend.

I'll make the change to $_agenix_generation, but I'm not enthused by the idea of tying it to the NixOS generation -- mostly because that'll still be complicated.

How do we get the generation? Reading the /nix/var/nix/profiles/system symlink isn't robust enough because it's possible to install the system under a different profile name (see nixos-rebuild(8) and search for --profile-name). What if this is on brand a new install? I doubt there's a generation to read / parse at that point.

Is this the approach that sops-nix uses? Intuitively I don't see too much of a reason to match it to the system generation as long as it works with rollbacks.

ryantm · 2021-11-16T03:48:17Z

Sounds fine to keep it separate I just guessing maybe the generation was easily available as an env var or something

This follows sops-nix's implementation, where it creates a `/run/secrets.d` ramfs mountpoint and a "generation" each time the activation script runs, and then symlinks `/run/secrets` to `/run/secrets.d/[generation]`.

There are some cases where it may be better or even required to have the secret be a file that is not a symlink. Setting age.secrets.some-secret.symlink = false; will disable the default functionality of symlinking secrets and instead just forcibly move them to their `path`.

cole-h · 2021-11-16T05:42:38Z

Is this the approach that sops-nix uses? Intuitively I don't see too much of a reason to match it to the system generation as long as it works with rollbacks.

I don't remember, but either way, it should work with rollbacks since the path to the encrypted secret is referenced from the system activation script. Even if you run it multiple times, it will just create a new generation with the expected decrypted secrets each time.

(I lied earlier -- I found time tonight to change the variable name hehe)

cole-h commented Feb 25, 2021

View reviewed changes

modules/age.nix Outdated Show resolved Hide resolved

cole-h force-pushed the symlink branch from d8bc440 to bbf8eb9 Compare February 25, 2021 18:05

cole-h changed the title ~~feat: symlink files into place~~ modules/age: symlink files into place Feb 25, 2021

cole-h force-pushed the symlink branch 2 times, most recently from 8944f50 to 28b3369 Compare February 26, 2021 18:31

cole-h commented Feb 27, 2021

View reviewed changes

modules/age.nix Outdated Show resolved Hide resolved

cole-h force-pushed the symlink branch 2 times, most recently from 42facc9 to b3480ac Compare March 1, 2021 22:10

asymmetric reviewed Mar 2, 2021

View reviewed changes

modules/age.nix Outdated Show resolved Hide resolved

cole-h force-pushed the symlink branch from b3480ac to d8808a6 Compare March 2, 2021 22:21

asymmetric reviewed Mar 23, 2021

View reviewed changes

cole-h force-pushed the symlink branch from 16a80c5 to b0480a5 Compare March 23, 2021 20:00

cole-h commented Mar 24, 2021

View reviewed changes

modules/age.nix Outdated Show resolved Hide resolved

cole-h force-pushed the symlink branch 3 times, most recently from 8e1c150 to 4991bc4 Compare May 10, 2021 18:22

ryantm requested changes May 10, 2021

View reviewed changes

modules/age.nix Outdated Show resolved Hide resolved

modules/age.nix Outdated Show resolved Hide resolved

modules/age.nix Outdated Show resolved Hide resolved

cole-h commented May 10, 2021

View reviewed changes

modules/age.nix Show resolved Hide resolved

cole-h force-pushed the symlink branch from 2035710 to 965c5eb Compare May 13, 2021 03:41

Gerschtli mentioned this pull request Nov 8, 2021

How to cleanup (aka remove) secret files after they removed from home-manager config? jordanisaacs/homeage#6

Closed

cole-h force-pushed the symlink branch from 965c5eb to 4b6c6ec Compare November 8, 2021 17:51

cole-h added 4 commits November 15, 2021 21:39

modules/age: symlink files into place

f816a0d

This follows sops-nix's implementation, where it creates a `/run/secrets.d` ramfs mountpoint and a "generation" each time the activation script runs, and then symlinks `/run/secrets` to `/run/secrets.d/[generation]`.

modules/age: remove old secrets generations

111754b

modules/age: /run/secrets -> /run/agenix

e538664

cole-h force-pushed the symlink branch from 4b6c6ec to 7bb0b5d Compare November 16, 2021 05:39

ryantm merged commit 7bb0b5d into ryantm:master Nov 20, 2021

cole-h deleted the symlink branch November 20, 2021 18:17

chvp mentioned this pull request Nov 20, 2021

Secrets owned by non-root users are no longer readable by that user #69

Closed

johnae added a commit to johnae/world that referenced this pull request Nov 21, 2021

agenix: /run/secrets -> /run/agenix, see: ryantm/agenix#27

0b27951

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

modules/age: symlink files into place #27

modules/age: symlink files into place #27

cole-h commented Feb 25, 2021 •

edited

Loading

cole-h commented Feb 27, 2021

cole-h commented Mar 2, 2021

asymmetric left a comment

asymmetric Mar 23, 2021

cole-h Mar 23, 2021 •

edited

Loading

kclejeune commented May 9, 2021 •

edited

Loading

cole-h commented May 10, 2021 •

edited

Loading

alarsyo commented Sep 18, 2021

kclejeune commented Oct 20, 2021

cole-h commented Oct 20, 2021

ryantm commented Oct 20, 2021

kclejeune commented Oct 20, 2021

ryantm commented Oct 20, 2021

kclejeune commented Oct 20, 2021

Gerschtli commented Nov 8, 2021

ryantm commented Nov 8, 2021

Gerschtli commented Nov 8, 2021 •

edited

Loading

cole-h commented Nov 8, 2021

ryantm commented Nov 8, 2021

cole-h commented Nov 8, 2021

ryantm commented Nov 10, 2021

cole-h commented Nov 16, 2021 •

edited

Loading

kclejeune commented Nov 16, 2021

ryantm commented Nov 16, 2021

cole-h commented Nov 16, 2021

modules/age: symlink files into place #27

modules/age: symlink files into place #27

Conversation

cole-h commented Feb 25, 2021 • edited Loading

cole-h commented Feb 27, 2021

cole-h commented Mar 2, 2021

asymmetric left a comment

Choose a reason for hiding this comment

asymmetric Mar 23, 2021

Choose a reason for hiding this comment

cole-h Mar 23, 2021 • edited Loading

Choose a reason for hiding this comment

kclejeune commented May 9, 2021 • edited Loading

cole-h commented May 10, 2021 • edited Loading

alarsyo commented Sep 18, 2021

kclejeune commented Oct 20, 2021

cole-h commented Oct 20, 2021

ryantm commented Oct 20, 2021

kclejeune commented Oct 20, 2021

ryantm commented Oct 20, 2021

kclejeune commented Oct 20, 2021

Gerschtli commented Nov 8, 2021

ryantm commented Nov 8, 2021

Gerschtli commented Nov 8, 2021 • edited Loading

cole-h commented Nov 8, 2021

ryantm commented Nov 8, 2021

cole-h commented Nov 8, 2021

ryantm commented Nov 10, 2021

cole-h commented Nov 16, 2021 • edited Loading

kclejeune commented Nov 16, 2021

ryantm commented Nov 16, 2021

cole-h commented Nov 16, 2021

cole-h commented Feb 25, 2021 •

edited

Loading

cole-h Mar 23, 2021 •

edited

Loading

kclejeune commented May 9, 2021 •

edited

Loading

cole-h commented May 10, 2021 •

edited

Loading

Gerschtli commented Nov 8, 2021 •

edited

Loading

cole-h commented Nov 16, 2021 •

edited

Loading