Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compose: support IMA/fsverity #1883

Closed
lucab opened this issue Aug 10, 2019 · 13 comments
Closed

compose: support IMA/fsverity #1883

lucab opened this issue Aug 10, 2019 · 13 comments

Comments

@lucab
Copy link
Contributor

lucab commented Aug 10, 2019

As an additional step in boot measurement, and in order to extend the chain of measurement to the userland, it would be interesting for rpm-ostree to attach IMA digests (i.e. security.ima xattr) to committed files.

In particular, it would be nice getting to a point where we are covering the individual files in initramfs. That should in theory allow us to bridge between a measuring bootloader and the real rootfs binaries (checked out from ostree).

I am not sure if appending xattrs would break some content integrity reference with RPM pristine content, plus some RPMs may already be coming with security.ima labeled files.
In that case, it could still be worth investigating labeling files that are not coming from RPMs, e.g. content injected via treefile postprocess.

@cgwalters
Copy link
Member

it would be interesting for rpm-ostree to attach IMA digests (i.e. security.ima xattr) to committed files.

Yes! Thanks for filing this. I would love to investigate this (part of this should be libostree work).
I recently was following with great interest this thread and I want to watch the recordings when they become available.

I am not sure if appending xattrs would break some content integrity reference with RPM pristine content

Oh but we're going to intercept that anyways 😉

plus some RPMs may already be coming with security.ima labeled files.

Not in Fedora at least. Although I think IBM (main IMA sponsor?) was working on that, you can find patches on the rpm-maint list. But it's not relevant for us because librpm isn't involved in actually writing files.

@cgwalters
Copy link
Member

@lucab I propose we move this to a fedora-coreos-tracker issue. I think it encompasses more things; it has an impact on the administrator experience beyond just rpm-ostree.

(Also, I think we should support "sealing" after Ignition has run - in true immutable infrastructure style, /etc would be read-only after that, and we'd try to strongly enforce it or so)

@lucab
Copy link
Contributor Author

lucab commented Sep 13, 2019

@cgwalters I'm fine to also have an umbrella issue in fcos-tracker to tie all the components and parts together. But I currently don't have all the required knowledge to see the whole picture, so I didn't open that (yet).

This ticket is just to scope the initial small step of "ensure that everything has a digest in xattr", either here or in libostree.

@cgwalters
Copy link
Member

I recently was following with great interest this thread and I want to watch the recordings when they become available.

https://www.youtube.com/watch?v=EmEymlA5Q5Q
was quite interesting indeed!

@cgwalters
Copy link
Member

See also https://github.com/keylime/keylime

@cgwalters
Copy link
Member

cgwalters commented Nov 11, 2019

I'm also investigating fs-verity, see:
ostreedev/ostree#1959
ostreedev/ostree#2269

The reasons to go for fs-verity over IMA are:

  • Files are actually read-only even to CAP_SYS_ADMIN
  • Verifies file chunks rather than whole thing, meaning reduced startup latency and avoids "read amplification" for files that are only partially read
  • More modern crypto
  • Relatedly, I believe fs-verity better handles "offline" attacks because it validates the in-memory state

That said, as the upstream fs-verity docs note, IMA could use fs-verity as a "backend"; I think that makes the most sense. Hopefully patches happen in that direction.

The downside of fs-verity is that we will want to patch core parts of userspace (e.g. systemd) to implement things like

ExecRequireFsVerity=true

(with a well-known kernel keyring value e.g.)

@cgwalters
Copy link
Member

However before we do anything like this I think we need to implement a basic "trusted boot" for FCOS that gets us into verified userspace so that we can have that userspace e.g. check fs-verity signatures. Most likely, I think what we should do is teach rpm-ostree how to optionally generate a combined kernel+initramfs image that is signed. See:

I think the systemd crew tested this in concert with sd-boot.

@cgwalters
Copy link
Member

@cgwalters cgwalters changed the title compose: consider attaching IMA digest when committing files compose: support verified userspace Nov 17, 2019
@cgwalters
Copy link
Member

Definitely some useful discussion in https://lwn.net/Articles/791863/ too.

@cgwalters
Copy link
Member

I still think it'd probably be worth adding fs-verity to xfs and enable it by default. Honestly it'd just be a tiny speed bump at best for most attackers but it'd be a useful step along the way to stronger security, and I do like the fact it actually makes files immutable beyond just the readonly bind mount.

That said...I think the only truly strong solution here is to go to a model where each node deploys a fully verified immutable filesystem tree for the OS that is signed by the end user's key. Which would be a radical change for us (both "core" CoreOS as well as e.g. OpenShift). For OpenShift we'd need to e.g. change the MCO to take the Ignition (plus kernel args) and generate a new ostree commit (or perhaps simpler to start really, use dm-verity like others are) and sign it with a key the admin provides and set everything up from Secure Boot to chain into that. This would be a nontrivial amount of work and also make things like "I want to test a kernel change on this one node" much harder (but that use case is exactly the same as an attacker wanting to persist on that node...)

cgwalters added a commit to cgwalters/ostree that referenced this issue Jan 10, 2021
At the time we added fsverity code to ostree, fsverity was
just a CLI tool; since then it has
gained a C shared library which wraps all the signature
bits and the enablement `ioctl()` conveniently.

This makes it much easier for us to support signatures, so
do so.  Note that at this time, because ostree doesn't define
a mechanism to transport fsverity signatures across repositories,
this is mostly only useful for locally-generated signatures.

The idea however is this is a starting point from which we can
build more support, including signature transport, remote keys,
etc.

In order to simplify things, drop support for "opportunistic"
use of fsverity.  In practice we expect people using this
to set it up fully, or not at all.  The "read only files"
aspect *is* useful, but complicated the code too much relative
to its benefit.

Also drop support for using fsverity for `/boot` for now; in
practice most things there are read by the bootloader,
which doesn't know about fsverity.  Instead those should
be covered by e.g. Secure Boot.  This ensures that
we only have one high level API `_ostree_tmpf_fsverity()`
that is invoked from the core commit path.

xref https://lwn.net/Articles/842002
xref ostreedev#1959
xref coreos/rpm-ostree#1883
cgwalters added a commit to cgwalters/ostree that referenced this issue Jan 10, 2021
At the time we added fsverity code to ostree, fsverity was
just a CLI tool; since then it has
gained a C shared library which wraps all the signature
bits and the enablement `ioctl()` conveniently.

This makes it much easier for us to support signatures, so
do so.  Note that at this time, because ostree doesn't define
a mechanism to transport fsverity signatures across repositories,
this is mostly only useful for locally-generated signatures.

The idea however is this is a starting point from which we can
build more support, including signature transport, remote keys,
etc.

In order to simplify things, drop support for "opportunistic"
use of fsverity.  In practice we expect people using this
to set it up fully, or not at all.  The "read only files"
aspect *is* useful, but complicated the code too much relative
to its benefit.

Also drop support for using fsverity for `/boot` for now; in
practice most things there are read by the bootloader,
which doesn't know about fsverity.  Instead those should
be covered by e.g. Secure Boot.  This ensures that
we only have one high level API `_ostree_tmpf_fsverity()`
that is invoked from the core commit path.

xref https://lwn.net/Articles/842002
xref ostreedev#1959
xref coreos/rpm-ostree#1883
cgwalters added a commit to cgwalters/ostree that referenced this issue Jan 16, 2021
At the time we added fsverity code to ostree, fsverity was
just a CLI tool; since then it has
gained a C shared library which wraps all the signature
bits and the enablement `ioctl()` conveniently.

This makes it much easier for us to support signatures, so
do so.  Note that at this time, because ostree doesn't define
a mechanism to transport fsverity signatures across repositories,
this is mostly only useful for locally-generated signatures.

The idea however is this is a starting point from which we can
build more support, including signature transport, remote keys,
etc.

In order to simplify things, drop support for "opportunistic"
use of fsverity.  In practice we expect people using this
to set it up fully, or not at all.  The "read only files"
aspect *is* useful, but complicated the code too much relative
to its benefit.

Also drop support for using fsverity for `/boot` for now; in
practice most things there are read by the bootloader,
which doesn't know about fsverity.  Instead those should
be covered by e.g. Secure Boot.  This ensures that
we only have one high level API `_ostree_tmpf_fsverity()`
that is invoked from the core commit path.

xref https://lwn.net/Articles/842002
xref ostreedev#1959
xref coreos/rpm-ostree#1883
cgwalters added a commit to cgwalters/ostree that referenced this issue Jan 17, 2021
At the time we added fsverity code to ostree, fsverity was
just a CLI tool; since then it has
gained a C shared library which wraps all the signature
bits and the enablement `ioctl()` conveniently.

This makes it much easier for us to support signatures, so
do so.  Note that at this time, because ostree doesn't define
a mechanism to transport fsverity signatures across repositories,
this is mostly only useful for locally-generated signatures.

The idea however is this is a starting point from which we can
build more support, including signature transport, remote keys,
etc.

In order to simplify things, drop support for "opportunistic"
use of fsverity.  In practice we expect people using this
to set it up fully, or not at all.  The "read only files"
aspect *is* useful, but complicated the code too much relative
to its benefit.

Also drop support for using fsverity for `/boot` for now; in
practice most things there are read by the bootloader,
which doesn't know about fsverity.  Instead those should
be covered by e.g. Secure Boot.  This ensures that
we only have one high level API `_ostree_tmpf_fsverity()`
that is invoked from the core commit path.

xref https://lwn.net/Articles/842002
xref ostreedev#1959
xref coreos/rpm-ostree#1883
@cgwalters cgwalters changed the title compose: support verified userspace compose: support IMA/fsverity Jan 22, 2021
cgwalters added a commit to cgwalters/ostree that referenced this issue Jan 24, 2021
At the time we added fsverity code to ostree, fsverity was
just a CLI tool; since then it has
gained a C shared library which wraps all the signature
bits and the enablement `ioctl()` conveniently.

This makes it much easier for us to support signatures, so
do so.  Note that at this time, because ostree doesn't define
a mechanism to transport fsverity signatures across repositories,
this is mostly only useful for locally-generated signatures.

The idea however is this is a starting point from which we can
build more support, including signature transport, remote keys,
etc.

In order to simplify things, drop support for "opportunistic"
use of fsverity.  In practice we expect people using this
to set it up fully, or not at all.  The "read only files"
aspect *is* useful, but complicated the code too much relative
to its benefit.

Also drop support for using fsverity for `/boot` for now; in
practice most things there are read by the bootloader,
which doesn't know about fsverity.  Instead those should
be covered by e.g. Secure Boot.  This ensures that
we only have one high level API `_ostree_tmpf_fsverity()`
that is invoked from the core commit path.

xref https://lwn.net/Articles/842002
xref ostreedev#1959
xref coreos/rpm-ostree#1883
cgwalters added a commit to cgwalters/ostree that referenced this issue Jan 25, 2021
At the time we added fsverity code to ostree, fsverity was
just a CLI tool; since then it has
gained a C shared library which wraps all the signature
bits and the enablement `ioctl()` conveniently.

This makes it much easier for us to support signatures, so
do so.  Note that at this time, because ostree doesn't define
a mechanism to transport fsverity signatures across repositories,
this is mostly only useful for locally-generated signatures.

The idea however is this is a starting point from which we can
build more support, including signature transport, remote keys,
etc.

In order to simplify things, drop support for "opportunistic"
use of fsverity.  In practice we expect people using this
to set it up fully, or not at all.  The "read only files"
aspect *is* useful, but complicated the code too much relative
to its benefit.

Also drop support for using fsverity for `/boot` for now; in
practice most things there are read by the bootloader,
which doesn't know about fsverity.  Instead those should
be covered by e.g. Secure Boot.  This ensures that
we only have one high level API `_ostree_tmpf_fsverity()`
that is invoked from the core commit path.

xref https://lwn.net/Articles/842002
xref ostreedev#1959
xref coreos/rpm-ostree#1883
@cgwalters
Copy link
Member

@jlebon
Copy link
Member

jlebon commented Jun 3, 2022

Should we close this one in favour of discussions in ostreedev/ostree#2609 and https://github.com/giuseppe/composefs ?

@cgwalters
Copy link
Member

Yes, closing as a dup of ostreedev/ostree#2867

@cgwalters cgwalters closed this as not planned Won't fix, can't repro, duplicate, stale May 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants