#20472 probably broke macOS docker build #20853
Comments
kYc0o
added
Type: bug
|
Calling @maribu and @mguetschow who touched this code recently 🙏 |
|
Interesting. Which version of docker are you using? What is the output of |
|
For now a workaround was to change the variable to the actual SHA256: diff --git a/makefiles/docker.inc.mk b/makefiles/docker.inc.mk
index 34ef41e71d..9e1ded2f29 100644
--- a/makefiles/docker.inc.mk
+++ b/makefiles/docker.inc.mk
@@ -5,7 +5,7 @@ DOCKER_TESTED_IMAGE_ID := bbd6bc053ac3eafb173a475e0439376db91b9a88f4b504e1bfa4e0
DOCKER_TESTED_IMAGE_REPO_DIGEST := 52ee7ae8ec4f9b8852aac15cf349d748484cd4d0732b6e030eddf4fff854e799
DOCKER_PULL_IDENTIFIER := docker.io/riot/riotbuild@sha256:$(DOCKER_TESTED_IMAGE_REPO_DIGEST)
-DOCKER_IMAGE_DEFAULT := sha256:$(DOCKER_TESTED_IMAGE_ID)
+DOCKER_IMAGE_DEFAULT := sha256:$(DOCKER_TESTED_IMAGE_REPO_DIGEST)
DOCKER_AUTO_PULL ?= 1
export DOCKER_IMAGE ?= $(DOCKER_IMAGE_DEFAULT)
export DOCKER_BUILD_ROOT ?= /data/riotbuild |
% docker --version
Docker version 27.2.0, build 3ab4256{
"Containers": "N/A",
"CreatedAt": "2024-08-28 14:59:31 +0200 CEST",
"CreatedSince": "8 days ago",
"Digest": "",
"ID": "52ee7ae8ec4f",
"Repository": "riot/riotbuild",
"SharedSize": "N/A",
"Size": "17.5GB",
"Tag": "<none>",
"UniqueSize": "N/A",
"VirtualSize": "17.45GB"
} |
|
OK, ID in the JSON is the same as the digest of the repository on your system. That is not how this works on Linux/x86_64 systems. E.g. we get (See the mismatch in I honestly don't know what went wrong when the docker guys decided that the same image will get a different SHA256, depending on whether the image is locally installed or sits in the docker hub. But I have even less sympathy with the docker guys not being consistent about this, but deciding to use the same SHA256 as in the docker hub for some systems when locally installed, but not for other systems. I wonder if it is the better approach to just check for both SHA256 locally and just detect what ID system is used to refer to local images, or whether we should inspect the system attributes and deduce from that what ID system is used. Do you know if |
|
podman is supported yeah, shall I try it? |
|
That would be nice. If |
It turns out that the ID mechanics of docker are even more crazy that realized before: On Linux (x86_64) they use a different SHA256 when referring to a locally installed image than when referring to the same image at dockerhub. On Mac OS (Apple Silicon), the use the repo SHA256 also when referring to the local image. Instead of increasing the complexity of the current solution even more by covering both cases, we now use `docker.io/riot/riotbuild@sha256:<SHA256_OF_DOCKERHUB_IMAGE>` to refer to a specific docker image, which hopefully works across systems. Instead of pulling the image explicitly, we now can rely on docker to do so automatically if the pinned image is not found locally. As a result, the knob to disable automatic pulling has been dropped. Fixes RIOT-OS#20853
It turns out that the ID mechanics of docker are even more crazy than realized before: On Linux (x86_64) they use a different SHA256 when referring to a locally installed image than when referring to the same image at dockerhub. On Mac OS (Apple Silicon), the use the repo SHA256 also when referring to the local image. Instead of increasing the complexity of the current solution even more by covering both cases, we now use `docker.io/riot/riotbuild@sha256:<SHA256_OF_DOCKERHUB_IMAGE>` to refer to a specific docker image, which hopefully works across systems. Instead of pulling the image explicitly, we now can rely on docker to do so automatically if the pinned image is not found locally. As a result, the knob to disable automatic pulling has been dropped. Fixes RIOT-OS#20853
It turns out that the ID mechanics of docker are even more crazy than realized before: On Linux (x86_64) they use a different SHA256 when referring to a locally installed image than when referring to the same image at dockerhub. On Mac OS (Apple Silicon), the use the repo SHA256 also when referring to the local image. Instead of increasing the complexity of the current solution even more by covering both cases, we now use `docker.io/riot/riotbuild@sha256:<SHA256_OF_DOCKERHUB_IMAGE>` to refer to a specific docker image, which hopefully works across systems. Instead of pulling the image explicitly, we now can rely on docker to do so automatically if the pinned image is not found locally. As a result, the knob to disable automatic pulling has been dropped. Fixes RIOT-OS#20853
It turns out that the ID mechanics of docker are even more crazy than realized before: On Linux (x86_64) they use a different SHA256 when referring to a locally installed image than when referring to the same image at dockerhub. On Mac OS (Apple Silicon), the use the repo SHA256 also when referring to the local image. Instead of increasing the complexity of the current solution even more by covering both cases, we now use `docker.io/riot/riotbuild@sha256:<SHA256_OF_DOCKERHUB_IMAGE>` to refer to a specific docker image, which hopefully works across systems. Instead of pulling the image explicitly, we now can rely on docker to do so automatically if the pinned image is not found locally. As a result, the knob to disable automatic pulling has been dropped. Fixes RIOT-OS#20853
|
I'm also on MacOS on ARM64 and it seems like this issue is not a general Docker/MacOS/ARM64 issue. I've been using the RIOT Docker build with an older Docker version (20.10.22) without any problems. Only after upgrading to the latest Docker version (27.2.0) this issue occurred. The linked PR already fixes it for me, so this additional information is not that important any more. Just wanted to provide some more context to this issue. |
|
Thanks for confirming the issue is now fixed by the PR. I have hit the merge button now, so that the issue should be solved in |
|
So by reading the info in docker/for-mac#2396 I've realised that I haven't added |
Description
It seems that #20472 broke the fetching/using the right docker image in a macOS (ARM64, Apple M3 Pro chip)
I'm trying to figure out what exactly is wrong (if the sha of the image or the name of the image).
Steps to reproduce the issue
Try to build RIOT with
BUILD_IN_DOCKER=1on macOSExpected results
It builds the binary using the docker image
Actual results
Versions
The text was updated successfully, but these errors were encountered: