root from an ostree-container

[cgwalters]

Today we have two "roots": the squashfs (used to generate the ISO and PXE) and the qcow2.

Once we productize https://github.com/ostreedev/ostree-rs-ext/#module-container-encapsulate-ostree-commits-in-ocidocker-images i.e. coreos/fedora-coreos-tracker#812 then our diskimage container could derive from it.

This would offer a lot of advantages; we could use it instead of the qcow2 as one root.

We can assume most people who want to mirror our disk images also want to mirror the os updates, so we'd get use container image layering for deduplication that way.

[cgwalters]

I've been thinking about this more and I think it basically requires that we ship the exact same version of qemu-img and mksquashfs in our container that were used by coreos-assembler.

Which, in practice is going to require some entanglement/interlock with that.

And further, raises the problem for RHCOS in that we are likely going to need to support a flow that uses a RHEL version of those tools. Which would be messy today - we'd need to do something like use "fcosa"¹ to generate the ostree and then take that data into a rhcosa² container image that generates the disk images. Then we ensure that when we build the dehydrated -images container we use those exact binaries. (Hmm, maybe to start we inject the versions of qemu-img and mksquashfs into the stream metadata?)

¹ Yep I just made that up: fedora coreos assembler
² rhel cosa

[cgwalters]

Alternatively...one other angle is to investigate using centos-stream for cosa. I commented on this one in a different bug. This would force cosa into being more obviously a "cross" tool. We'd still have the ability to rev our tooling quickly without being forced to wait on shipping it in RHEL. But it'd have a longer lifecycle, and rev tools like qemu-img and mksquashfs more slowly.

So I guess an example downside would be if e.g. we wanted to flip on using zstd for mksquashfs in FCOS first.