image-layout: ./refs/ -> index.json

From the 2017-01-25 call we agreed to keep the manifest list as the
entry point for accessing objects. The index concept is only a more
generic use of the manifest-list.
#438 (comment)

This change uses the manifest-list as an index that allows
implementations to work around needing to walk/traverse the `./refs/`
directory.

As the `./refs/` directory was REQUIRED to exist, with its removal this
`index.json` is REQUIRED to be present.

This includes validating the image-layout example.

This also makes `validate-examples` automatically update `schema-fs` if
there have been changes to the JSON schema.

Obsoletes #438

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>

Makefile

@@ -75,12 +75,14 @@ $(OUTPUT_DIRNAME)/$(DOC_FILENAME).html: $(DOC_FILES) $(FIGURE_FILES)
 	ls -sh $(shell readlink -f $@)
 endif
 
-validate-examples:
+validate-examples: schema/fs.go
 	go test -run TestValidate ./schema
 
-schema-fs:
+schema/fs.go: $(wildcard schema/*.json) schema/gen.go
+	cd schema && printf "%s\n\n%s\n" "$$(cat ../.header)" "$$(go generate)" > fs.go
+
+schema-fs: schema/fs.go
 	@echo "generating schema fs"
-	@cd schema && printf "%s\n\n%s\n" "$$(cat ../.header)" "$$(go generate)" > fs.go
 
 check-license:
 	@echo "checking license headers"
@@ -90,7 +92,7 @@ lint:
 	@echo "checking lint"
 	@./.tool/lint
 
-test:
+test: schema/fs.go
 	go test -race -cover $(shell go list ./... | grep -v /vendor/)
 
 img/%.png: img/%.dot
@@ -129,4 +131,5 @@ clean:
 	clean \
 	lint \
 	docs \
-	test
+	test \
+	schema-fs

annotations.md

@@ -21,3 +21,4 @@ This specification defines the following annotation keys, intended for but not l
 * **org.opencontainers.homepage** URL to find more information on the image (string, a URL with scheme HTTP or HTTPS)
 * **org.opencontainers.documentation** URL to get documentation on the image (string, a URL with scheme HTTP or HTTPS)
 * **org.opencontainers.source** URL to get source code for the binary files in the image (string, a URL with scheme HTTP or HTTPS)
+* **org.opencontainers.ref.name** Name of the reference (string)

config.md

@@ -229,4 +229,4 @@ Here is an example image configuration JSON document:
 ```
 
 [rfc3339-s5.6]: https://tools.ietf.org/html/rfc3339#section-5.6
-[runtime-platform]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc2/config.md#platform
+[runtime-platform]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc3/config.md#platform

image-layout.md

@@ -3,47 +3,45 @@
 The OCI Image Layout is a slash separated layout of OCI content-addressable blobs and [location-addressable](https://en.wikipedia.org/wiki/Content-addressable_storage#Content-addressed_vs._location-addressed) references (refs).
 This layout MAY be used in a variety of different transport mechanisms: archive formats (e.g. tar, zip), shared filesystem environments (e.g. nfs), or networked file fetching (e.g. http, ftp, rsync).
 
-Given an image layout and a ref, a tool can create an [OCI Runtime Specification bundle](https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc2/bundle.md) by:
+Given an image layout and a ref, a tool can create an [OCI Runtime Specification bundle](https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc3/bundle.md) by:
 
 * Following the ref to find a [manifest](manifest.md#image-manifest), possibly via a [manifest list](manifest-list.md#manifest-list)
 * [Applying the filesystem layers](layer.md#applying) in the specified order
-* Converting the [image configuration](config.md) into an [OCI Runtime Specification `config.json`](https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc2/config.md)
+* Converting the [image configuration](config.md) into an [OCI Runtime Specification `config.json`](https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc3/config.md)
 
 # Content
 
-The image layout MUST contain two top level directories:
-
-- `blobs` contains content-addressable blobs.
-  A blob has no schema and should be considered opaque.
-- `refs` contains [descriptors][descriptors].
-  Commonly pointing to an [image manifest](manifest.md#image-manifest) or an [image manifest list](manifest-list.md#oci-image-manifest-list-specification).
-
-Both `blobs` and `refs` MAY be empty.
-
-The image layout MUST also contain an `oci-layout` file:
-
-- It MUST be a JSON object
-- It MUST contain an `imageLayoutVersion` field
-    - The `imageLayoutVersion` value will align with the OCI Image Specification version at the time changes to the layout are made, and will pin a given version until changes to the layout are required
-- It MAY include additional fields
+The image layout is as follows:
+
+- `blobs` directory
+    - Contains content-addressable blobs
+    - A blob has no schema and should be considered opaque
+    - Directory MUST exist and MAY be empty
+    - See [blobs](#blobs) section
+- `oci-layout` file
+    - It MUST exist
+    - It MUST be a JSON object
+    - It MUST contain an `imageLayoutVersion` field
+    - See [oci-layout file](#oci-layout-file) section
+    - It MAY include additional fields
+- `index.json` file
+    - It MUST exist
+    - It MUST be a JSON object
+    - It MUST have the base properties of [manifest-list](manifest-list.md).
+    - See [index.json](#indexjson-file) section
 
 ## Example Layout
 
 This is an example image layout:
 
 ```
 $ cd example.com/app/
-$ find .
-.
-./blobs
-./blobs/sha256/e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f
-./blobs/sha256/afff3924849e458c5ef237db5f89539274d5e609db5db935ed3959c90f1f2d51
-./blobs/sha256/9b97579de92b1c195b85bb42a11011378ee549b02d7fe9c17bf2a6b35d5cb079
-./blobs/sha256/5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270
+$ find . -type f
+./index.json
 ./oci-layout
-./refs
-./refs/v1.0
-./refs/stable-release
+./blobs/sha256/3588d02542238316759cbf24502f4344ffcc8a60c803870022f335d1390c13b4
+./blobs/sha256/4b0bc1c4050b03c95ef2a8e36e25feac42fd31283e8c30b3ee5df6b043155d3c
+./blobs/sha256/7968321274dc6b6171697c33df7815310468e694ac5be0ec03ff053bb135e768
 ```
 
 Blobs are named by their contents:
@@ -53,33 +51,6 @@ $ shasum -a 256 ./blobs/sha256/afff3924849e458c5ef237db5f89539274d5e609db5db935e
 afff3924849e458c5ef237db5f89539274d5e609db5db935ed3959c90f1f2d51 ./blobs/sha256/afff3924849e458c5ef237db5f89539274d5e609db5db935ed3959c90f1f2d51
 ```
 
-## Refs
-
-Object names in the `refs` subdirectories MUST NOT include characters outside of the set of "A" to "Z", "a" to "z", "0" to "9", the hyphen `-`, the dot `.`, and the underscore `_`.
-
-No semantic restriction is given for object names in the `refs` subdirectory.
-Each object in the `refs` subdirectory MUST be of type `application/vnd.oci.descriptor.v1+json`.
-In general the `mediaType` of this [descriptor][descriptors] object will be either `application/vnd.oci.image.manifest.list.v1+json` or `application/vnd.oci.image.manifest.v1+json` although future versions of the spec MAY use a different mediatype.
-
-**Implementor's Note:**
-A common use case of refs is representing "tags" for a container image.
-For example, an image may have a tag for different versions or builds of the software.
-In the wild you often see "tags" like "v1.0.0-vendor.0", "2.0.0-debug", etc.
-Those tags will often be represented in an image-layout repository with matching refs names like "v1.0.0-vendor.0", "2.0.0-debug", etc.
-
-### Example Ref
-
-This is an example `v1.0` ref with a manifest-list descriptor:
-
-```
-$ cat ./refs/v1.0 | jq
-{
-  "size": 4096,
-  "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f",
-  "mediaType": "application/vnd.oci.image.manifest.list.v1+json"
-}
-```
-
 ## Blobs
 
 Object names in the `blobs` subdirectories are composed of a directory for each hash algorithm, the children of which will contain the actual content.
@@ -156,4 +127,78 @@ $ cat ./blobs/sha256/e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7f
 [tar stream]
 ```
 
+## oci-layout file
+
+This JSON object serves as a marker for the base of an Open Container Image Layout and to provide the version of the image-layout in use.
+The `imageLayoutVersion` value will align with the OCI Image Specification version at the time changes to the layout are made, and will pin a given version until changes to the image layout are required.
+
+### oci-layout Example
+
+```json
+{
+    "imageLayoutVersion": "1.0.0"
+}
+```
+
+## index.json file
+
+This REQUIRED file is the entry point for references and descriptors of the image-layout.
+The [manifest-list](manifest-list.md) is a multi-descriptor entry point.
+
+This index provides an established path (`/index.json`) to have an entry point for an image-layout and to discover auxiliary descriptors.
+
+No semantic restriction is given for the "org.opencontainers.ref.name" annotation of descriptors.
+In general the `mediaType` of each [descriptor][descriptors] object in the `manifests` field will be either `application/vnd.oci.image.manifest.list.v1+json` or `application/vnd.oci.image.manifest.v1+json`
+Future versions of the spec MAY use a different mediatype (i.e. a new versioned format).
+An encountered `mediaType` that is unknown SHOULD be safely ignored.
+
+
+**Implementor's Note:**
+A common use case of descriptors with a "org.opencontainers.ref.name" annotation is representing a "tag" for a container image.
+For example, an image may have a tag for different versions or builds of the software.
+In the wild you often see "tags" like "v1.0.0-vendor.0", "2.0.0-debug", etc.
+Those tags will often be represented in an image-layout repository with matching "org.opencontainers.ref.name" annotations like "v1.0.0-vendor.0", "2.0.0-debug", etc.
+
+
+### Index Example
+
+```json,title=Manifest%20List&mediatype=application/vnd.oci.image.manifest.list.v1%2Bjson
+{
+  "schemaVersion": 2,
+  "manifests": [
+    {
+      "mediaType": "application/vnd.oci.image.manifest.v1+json",
+      "size": 7143,
+      "digest": "sha256:0228f90e926ba6b96e4f39cf294b2586d38fbb5a1e385c05cd1ee40ea54fe7fd",
+      "annotations": {
+        "org.opencontainers.ref.name": "stable-release"
+      }
+    },
+    {
+      "mediaType": "application/vnd.oci.image.manifest.v1+json",
+      "size": 7143,
+      "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f",
+      "annotations": {
+        "org.opencontainers.ref.name": "v1.0"
+      }
+    },
+    {
+      "mediaType": "application/xml",
+      "size": 7143,
+      "digest": "sha256:b3d63d132d21c3ff4c35a061adf23cf43da8ae054247e32faa95494d904a007e",
+      "annotations": {
+        "org.freedesktop.specifications.metainfo.version": "1.0",
+        "org.freedesktop.specifications.metainfo.type": "AppStream"
+      }
+    }
+  ],
+  "annotations": {
+    "com.example.index.revision": "r124356"
+  }
+}
+```
+
+This illustrates an index that provides two named manifest references and an auxilary mediatype for this image layout.
+
+
 [descriptors]: ./descriptor.md

manifest-list.md

@@ -11,7 +11,9 @@ For the media type(s) that this document is compatible with, see the [matrix][ma
 - **`schemaVersion`** *int*
 
   This REQUIRED property specifies the image manifest schema version.
-  For this version of the specification, this MUST be `2` to ensure backward compatibility with older versions of Docker. The value of this field will not change. This field MAY be removed in a future version of the specification.
+  For this version of the specification, this MUST be `2` to ensure backward compatibility with older versions of Docker.
+  The value of this field will not change.
+  This field MAY be removed in a future version of the specification.
 
 - **`mediaType`** *string*
 
@@ -103,5 +105,5 @@ For the media type(s) that this document is compatible with, see the [matrix][ma
 }
 ```
 
-[runtime-platform2]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc2/config.md#platform
+[runtime-platform2]: https://github.com/opencontainers/runtime-spec/blob/v1.0.0-rc3/config.md#platform
 [matrix]: media-types.md#compatibility-matrix

schema/content-descriptor.json

@@ -23,6 +23,10 @@
         "type": "string",
         "format": "uri"
       }
+    },
+    "annotations": {
+      "id": "https://opencontainers.org/schema/image/descriptor/annotations",
+      "$ref": "defs-image.json#/definitions/annotations"
     }
   },
   "required": [

schema/defs-image.json

@@ -17,8 +17,7 @@
       "required": [
         "mediaType",
         "size",
-        "digest",
-        "platform"
+        "digest"
       ],
       "properties": {
         "mediaType": {
@@ -78,6 +77,10 @@
               }
             }
           }
+        },
+        "annotations": {
+          "id": "https://opencontainers.org/schema/image/descriptor/annotations",
+          "$ref": "#/definitions/annotations"
         }
       }
     },