Support ostree-format: oci in image.yaml

Part of implementing coreos/fedora-coreos-tracker#812

A whole lot of the story of coreos-assembler is threaded
with the tension between ostree and disk images.  They
have fundamentally different tradeoffs.  And now I'm trying
to add container images to the mix.

The idea of capturing an ostree repo in archive mode as a tarball
is a cosa invention.  We don't actually ship anything that way.

The proposal in the above linked issue is to "productize" support
for shipping ostree-in-container, because containers are just
slightly fancy tarballs.

This patch adds support for:
`echo 'ostree-format: oci' >> image.yaml`
in the config git.

When enabled, the `images/ostree` is replaced with an `oci-archive`
format of an "ostree-in-container", which we might shorten to
`ostcontainer` or so.  The code is updated to call out to
rpm-ostree's latest (really ostree-rs-ext's latest) code
to perform the export and import.

We're not making it the default yet, but I'd like to potentially
e.g. switch the FCOS `next` stream or so.

The next step after this lands is to add separate code in the
pipeline to push the image to a registry.
There's also a *lot* of deduplication/rationalization to
come later around `cosa upload-oscontainer` etc.

src/cmd-build

@@ -374,8 +374,7 @@ if [ ! -f /lib/coreos-assembler/.clean ]; then
     src_location="bind mount"
 fi
 
-# And create the ostree repo tarball containing the commit
-ostree_tarfile_path=${name}-${buildid}-ostree.${basearch}.tar
+# And create the ostree repo export containing the commit
 ostree_tarfile_sha256=
 if [ "${commit}" == "${previous_commit}" ] && \
     [ -f "${previous_builddir}/${previous_ostree_tarfile_path}" ]; then
@@ -386,19 +385,30 @@ if [ "${commit}" == "${previous_commit}" ] && \
         ostree_tarfile_sha256=
     fi
 else
-    ostree init --repo=repo --mode=archive
-    # Pass the ref if it's set
-    # shellcheck disable=SC2086
-    if ! ostree pull-local --repo=repo "${tmprepo}" "${buildid}" ${ref}; then
-        echo '(maybe https://github.com/coreos/coreos-assembler/issues/972 ?)'
-        exit 1
-    fi
-    # Don't compress; archive repos are already compressed individually and we'd
-    # gain ~20M at best. We could probably have better gains if we compress the
-    # whole repo in bare/bare-user mode, but that's a different story...
-    tar -cf "${ostree_tarfile_path}".tmp -C repo .
+    ostree_format=$(jq -r '.["ostree-format"]' < "${image_json}")
+    case "${ostree_format}" in
+        null|tar)
+            ostree_tarfile_path=${name}-${buildid}-ostree.${basearch}.tar
+            ostree init --repo=repo --mode=archive
+            # Pass the ref if it's set
+            # shellcheck disable=SC2086
+            if ! ostree pull-local --repo=repo "${tmprepo}" "${buildid}" ${ref}; then
+                echo '(maybe https://github.com/coreos/coreos-assembler/issues/972 ?)'
+                exit 1
+            fi
+            # Don't compress; archive repos are already compressed individually and we'd
+            # gain ~20M at best. We could probably have better gains if we compress the
+            # whole repo in bare/bare-user mode, but that's a different story...
+            tar -cf "${ostree_tarfile_path}".tmp -C repo .
+            rm -rf repo
+            ;;
+        oci)
+            ostree_tarfile_path=${name}-${buildid}-ostcontainer.${basearch}.ociarchive
+            rpm-ostree ex-container 'export' --repo="${tmprepo}" "${buildid}" oci-archive:${ostree_tarfile_path}.tmp
+            ;;
+        *) fatal "Unknown ostree-format: ${ostree_format}"
+    esac
     /usr/lib/coreos-assembler/finalize-artifact "${ostree_tarfile_path}"{.tmp,}
-    rm -rf repo
 fi
 
 if [ -z "${ostree_tarfile_sha256:-}" ]; then

src/cosalib/cmdlib.py

@@ -249,10 +249,19 @@ def import_ostree_commit(repo, buildpath, buildmeta, force=False):
         return
 
     # extract in a new tmpdir inside the repo itself so we can still hardlink
-    with tempfile.TemporaryDirectory(dir=repo) as d:
-        subprocess.check_call(['tar', '-C', d, '-xf', tarfile])
-        subprocess.check_call(['ostree', 'pull-local', '--repo', repo,
-                               d, commit])
+    if tarfile.endswith('.tar'):
+        with tempfile.TemporaryDirectory(dir=repo) as d:
+            subprocess.check_call(['tar', '-C', d, '-xf', tarfile])
+            subprocess.check_call(['ostree', 'pull-local', '--repo', repo,
+                                   d, commit])
+    elif tarfile.endswith('.ociarchive'):
+        # We do this in two stages, because right now ex-container only writes to
+        # non-archive repos.  Also, we need sudo to write to `repo-build`, though
+        # it might be good to change this by default.
+        build_repo = os.path.join(repo, '../../cache/repo-build')
+        subprocess.check_call(['sudo', 'rpm-ostree', 'ex-container', 'import', '--repo', build_repo,
+                               '--write-ref', buildmeta['buildid'], 'oci-archive:' + tarfile])
+        subprocess.check_call(['sudo', 'ostree', f'--repo={repo}', 'pull-local', build_repo, buildmeta['buildid']])
 
 
 def get_basearch():

tests/test_cosalib_cmdlib.py

@@ -122,7 +122,7 @@ def __call__(self, *args, **kwargs):
                     'ostree', 'init', '--repo', tmpdir, '--mode=archive']
             if self.check_call_count == 1:
                 assert args[0][0:2] == ['tar', '-C']
-                assert args[0][3:5] == ['-xf', './tarfile']
+                assert args[0][3:5] == ['-xf', './tarfile.tar']
             if self.check_call_count == 2:
                 assert args[0][0:4] == [
                     'ostree', 'pull-local', '--repo', tmpdir]
@@ -142,7 +142,7 @@ def monkeyspcall(*args, **kwargs):
         'ostree-commit': 'commit',
         'images': {
             'ostree': {
-                'path': 'tarfile'
+                'path': 'tarfile.tar'
             }
         }
     }