transport oci layouts instead of registry backends

[joshrwolf]

transporting content using the non-standard distribution backend provided a lot of convenience for serving, but it wastes the potential of the OCI Layout spec for all sorts of other things.

Evaluate transporting oci layouts instead, and then transferring into registry content at serve time or something similar

[nikkelma]

This would be part of a much larger conversation around in-transit formats alongside of disk requirements at runtime.

The benefits of using the distribution backend for transit allowed an package taking up 100GB on disk (uncompressed) to only require 100GB of disk space in the air gap.

In contrast, an OCI layout taking up 75G of space (uncompressed) on disk representing a distribution backend that takes up 100GB of space when serving the OCI registry needs 175GB of peak disk space, since there will be a non-trivial time period where the full OCI layout and the full distribution backend will exist on disk together.

If prioritizing disk space requirements, using the same format in the compressed archives as what the hauler OCI server will use at runtime is required. I'm unsure what options there are that allow for transparent inspection of the archive along with direct serving of a distribution-powered OCI registry server.