Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't see all architectures associated with a tag #130

Closed
bjj opened this issue May 21, 2020 · 6 comments · Fixed by #134
Closed

Can't see all architectures associated with a tag #130

bjj opened this issue May 21, 2020 · 6 comments · Fixed by #134
Assignees

Comments

@bjj
Copy link

bjj commented May 21, 2020

Just installed the UI. It was super easy with the REGISTRY_URL once I understood what it meant. The ability to put the registry on a private network and have the UI proxy /v2/ so it's all nice and tidy is fantastic.

Bug description

I'm building multi-platform images using docker buildx like this:

 docker buildx build --platform linux/amd64,linux/arm/v7 -t registry/name . --push

That is an all-in-one build/tag/push operation that results in an image I can run on arm64 or Raspberry Pi.

The UI shows the image and the (only) tag ("latest"). If I click history I see the amd64 version. Is there some way to see the arm/v7 version as well?

How to Reproduce

Use buildx as described to make a multi-platform image, then inspect with UI.

Expected behavior

Some indication that multiple architectures are supported by the tag. Some way to navigate them?

System information

  • Docker registry UI:
    • Version: v1.4.8
    • Server: docker
    • Docker version: 18.09.8
    • Docker registry ui tag: static
    • OS/Arch: amd64
    • Tools: docker-compose, docker buildx
@Joxit
Copy link
Owner

Joxit commented May 24, 2020

Hi @bjj and thank you for using my project and your issue 👍.

First of all, be careful, you are using an experimental docker feature and docker engine and docker registry servers do not evolve synchronously.

When you create a multi-arch image, here is what's going on:

  1. You are creating your two images
  2. You will export your first image amd64 (note the exporting config sha256)
 => => exporting layers
 => => exporting manifest sha256:e43d69ddc3d6d6f62f14d06ccacd528b24d671014b7a11e7dbde81a35c9a46fe
 => => exporting config sha256:13fed3dd0c000a1606eb29eafd20ac3e71751a09469a4c747b3cc6ea2e688414
  1. You will export your second image arm (note the exporting config sha256)
 => => exporting manifest sha256:3c883a759f83d9293f39ec036b017993821c8dfbeaee0b299003e5f86bd5f138
 => => exporting config sha256:c1f5bd20bd36aed176177c69a1b3491510b3b9ce3c6e30d5de00dd90826632ce
  1. You will create a manifest list that will contains the two previous manifests
 => => exporting manifest list sha256:53c9df714e3f28421c49574801c33865cf42fa2c15c1b02104dce20d4d1967b3
  1. You go to the UI and select your image, it will call the docker-registry API
  2. The first request is the tag list https://registry/v2/image-name/tags/list, the response is something like
{"name":"image-name","tags":["latest"]}
  1. The second request is to get the manifest from the tag https://registry/v2/image-name/manifests/latest and the response is something like
{
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "schemaVersion": 2,
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "digest": "sha256:13fed3dd0c000a1606eb29eafd20ac3e71751a09469a4c747b3cc6ea2e688414",
      "size": 6147
   },
   "layers": [ "more stuff here" ]
}

As you can see, the digest is 13fe... and this is the amd64 config sha256 !

  1. Now it will call the previous digest https://registry/v2/image-name/blobs/sha256:13fed3dd0c000a1606eb29eafd20ac3e71751a09469a4c747b3cc6ea2e688414 and we will get the amd64 history
{
  "architecture": "amd64",
  ...
}

While doing my research (and writing this), I saw that it is possible to get the list of architectures via the manifest list (in my example it was sha256:53c9df714e3f28421c49574801c33865cf42fa2c15c1b02104dce20d4d1967b3)

The idea here is to do the manifest list call between 7 and 8 🤔 but I don't know yet where I could display it in the interface 🤔

This functionality may require major changes to the interface. It could take a long time, but it is still possible. I would probably need your feedback when I start this feature, will it be possible?

@bjj
Copy link
Author

bjj commented May 24, 2020

Thanks for the detailed analysis. I'm very new to docker, so I learned a lot.

The second request is to get the manifest from the tag https://registry/v2/image-name/manifests/latest and the response is something like ...

I think this must be the difference. If I execute docker pull from Raspberry Pi and I look at the registry HTTP logs, it fetches the "latest" manifest, and then immediately fetches the exact manifest (by SHA) of the arm/v7 image. Then it fetches the blobs. The length of the returned manifest (which is in the log) is actually much shorter than what I get if I just fetch it with a browser.

As far as I can tell, this is controlled by the Accept header in the HTTP request (which I don't have from the logs). However, by experimenting, I can see that application/vnd.docker.distribution.manifest.list.v2+json returns something very useful: A list of manifests like this:

{
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "schemaVersion": 2,
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "digest": "sha256:8f69f6f30e4b64499de9f657acb2d80e0f4f71540cd368ceb8da7ed933012002",
         "size": 943,
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "digest": "sha256:40d9922185d39daa2ebce26a920899c9236b35741f7dfe5d4d12187664ff7300",
         "size": 943,
         "platform": {
            "architecture": "arm",
            "os": "linux",
            "variant": "v7"
         }
      }
   ]
}

@Joxit
Copy link
Owner

Joxit commented May 25, 2020

Oh yes! Your are right, application/vnd.docker.distribution.manifest.list.v2+json do the job ! This will be easier than expected ! Thanks 😄
Your response looks like mine when I do the request with the manifest list sha256:53c9df714e3f28421c49574801c33865cf42fa2c15c1b02104dce20d4d1967b3

The drawbacks with the header list is that I'm losing the layers key from the response... I use this to build the size of the Image. But I can do one request per architecture with the digest from the list. (The size in the manifest list is not the size of the image).

The docker client header list is

=> User-Agent: docker/19.03.9 go/go1.13.10 git-commit/9d988398e7 kernel/4.19.0-9-amd64 os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.9 \(linux\))
=> Accept: application/vnd.docker.distribution.manifest.v2+json
=> Accept: application/vnd.docker.distribution.manifest.list.v2+json
=> Accept: application/vnd.oci.image.index.v1+json
=> Accept: application/vnd.oci.image.manifest.v1+json
=> Accept: application/vnd.docker.distribution.manifest.v1+prettyjws
=> Accept: application/json
=> Accept-Encoding: gzip

Okay, now I have to think about the new interface to include this 🤔

@Joxit
Copy link
Owner

Joxit commented May 25, 2020

If the list of architectures was in the history page, would you like it?

@bjj
Copy link
Author

bjj commented May 25, 2020

Could the list of architectures be "tabs" at the top that let you select what you're looking at?

@Joxit
Copy link
Owner

Joxit commented May 25, 2020

tabs at the top of the tag history ? Yes absolutely !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging a pull request may close this issue.

2 participants