Add ARM64 Docker image support

[TheSilkky]

Adds support for a ARM64 Docker image using cross compiling with BlackDex/rust-musl. Uses my Dockerfile at TheSilkky/lemmy-docker with minimal changes to support debug builds and cargo features. I wasn't able to test this Dockerfile specifically but it should work since it's pretty similar to my images.

[TheSilkky]

I'll test the image build locally

[TheSilkky]

I successfully built the image locally

[TheSilkky]

By default the BlackDex image uses PostgreSQL Lib v11 and OpenSSL v1.1. It may be a good idea to use the OpenSSL 3 images and use PostgreSQL v15 instead with this environment variable: PQ_LIB_DIR="/usr/local/musl/pq15/lib".

[dessalines]

Works well on my dev machine. I've created a tag to start a woodpecker build, so we can see if this really works: https://woodpecker.join-lemmy.org/repos/129/pipeline/1889

[dessalines]

The apt update command fails for some reason: https://woodpecker.join-lemmy.org/repos/129/pipeline/1889/21

[TheSilkky]

The apt update command fails for some reason: https://woodpecker.join-lemmy.org/repos/129/pipeline/1889/21

I don't think this is an issue with the Dockerfile, probably something with the ci or maybe it was just random. I'm not sure.

[TheSilkky]

@dessalines Can you try running the build again?

[dessalines]

Started here: https://woodpecker.join-lemmy.org/repos/129/pipeline/1984

[TheSilkky]

@dessalines Did the build work? I can't see it for some reason.

[Nutomic]

The build timed out before it even reached the docker step. I tried it locally with the following command and it failed:

$ sudo docker build . -f docker/Dockerfile -t lemmy-arm --platform linux/arm643.2s
=> ERROR [stage-5 2/4] RUN apk add --no-cache     ca-certificates                                                                                     0.7s
------
> [stage-5 2/4] RUN apk add --no-cache     ca-certificates:
0.341 exec /bin/sh: exec format error
------
Dockerfile:99
--------------------
98 |
99 | >>> RUN apk add --no-cache \
100 | >>>     ca-certificates
101 |
--------------------
ERROR: failed to solve: process "/bin/sh -c apk add --no-cache     ca-certificates" did not complete successfully: exit code: 1

[TheSilkky]

@Nutomic You need to use buildx to build the image. Try this:
docker buildx build -f docker/Dockerfile -t lemmy-arm --platform linux/arm64 .

EDIT: Also make sure you have qemu on your system, it's needed for the last stage.

[TheSilkky]

Oops you'll probably also need qemu-user-static. You can install with your package manager on most systems. If you can't, you can run docker run --rm --privileged multiarch/qemu-user-static --reset -p yes which installs the latest version.

[TheSilkky]

I'm running a test build in GitHub actions: https://github.com/TheSilkky/lemmy/actions/runs/5795677444/job/15707886913

I'm using this workflow, I haven't used woodpecker before so I won't be able to help much with that, but maybe this could give an idea of how to set it up idk.

I'm pretty confident that the image works at this point, I think other issues are environmental.

[Nutomic]

I installed those packages and restarted Docker but it still gave the same error. Then tried the docker run command and it worked with that. That dependency definitely needs to be documented in the comment/readme file.

And that qemu dep is also necessary on all builders, right? We are running the builders from docker-compose.yml, so it seems easiest to add the qemu image to that file as well and avoid any separate install steps.

[TheSilkky]

QEMU isn't required for all builders, only when you're building for a different arch than the host. In the case of docker-compose.yml QEMU isn't necessary at all because you'd only be building for the host systems arch anyway. With wookpecker it's included with plugin-docker-buildx.

[dessalines]

Testing it again here: https://woodpecker.join-lemmy.org/repos/129/pipeline/2184

[TheSilkky]

@dessalines Shit it failed again, it didn't even get to the docker build. Is there anyway we can only run the docker stage?

[dessalines]

@TheSilkky move the docker publish stuff above all the tests in .woodpecker.yml, then once we get it working, we can move it down again.

https://woodpecker.join-lemmy.org/repos/129/pipeline/2195

[dessalines]

Thanks, I'll start a new one depending on whether the current build fails.

[dessalines]

Started: https://woodpecker.join-lemmy.org/repos/129/pipeline/2200/2

[TheSilkky]

The image seemed to build successfully but it seems there was issue while pushing, maybe something to do with that warning about git or maybe a network issue??

EDIT: Oh wait you wouldn't want it to push anyway yet, right?? So this isn't an issue then.

[dessalines]

Hit a weird Context deadline error, but the AMD64 build seemed to work fine. Maybe the arm platform isn't setting DNS right? https://stackoverflow.com/questions/67057789/error-on-context-deadline-with-docker-whats-the-meaning

I suggest trying to remove the AMD64 platform from woodpecker, so we can isolate the arm64 build and try again.

[TheSilkky]

At this point Docker is just pushing the layers to the registry, it's running directly on the host system, there's no virtualization or anything going on at this point. If it's a network issue it'll effect both builds the same, it's using the hosts network configuration.

Idk though, in my experience with docker sometimes shit like this just happens, I say we should just run it again to be sure that this wasn't just a random unrelated network issue.

[dessalines]

K I started running it again. It looks like it did correctly do the arm64 builds tho: https://hub.docker.com/r/dessalines/lemmy/tags

Just not sure why that build step is failing.

[joshdick]

One possibly small clue is that the publish_release_docker step reports as having taken 30:30 but the last log before the context deadline exceeded shows an elapsed time of 1749s, or 29:09. So something is seemingly hanging for over a minute after the layers finish pushing, assuming that either of those numbers can be trusted.

[dessalines]

Trying again: https://woodpecker.join-lemmy.org/repos/129/pipeline/2225/1

[dessalines]

Ugh, keeps failing. I'll try again: https://woodpecker.join-lemmy.org/repos/129/pipeline/2300/1

[TheSilkky]

Huh, it worked. But as far as I can tell nothing has really changed, so I'm guessing this time we just got lucky. I think it's network issues, given that it times out while pushing the layers, but who knows. What OS are the runners using?

[dessalines]

Most of the runners are on ubuntu, and all of them have buildkit. But since this passed I'd say its ready to be merged.

[Nutomic]

I understand that this step mentioned above should be added to woodpecker config, or is it not necessary anymore?

docker_buildx_prep:
  image: woodpeckerci/plugin-docker-buildx
  volumes:
    - /var/run/docker.sock:/var/run/docker.sock
  commands:
    - docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
  when:
    event: tag

[TheSilkky]

@Nutomic I don't think it's really necessary. It's just something that could be added if qemu related issues come up, which I don't think will happen on the runners.

[joshdick]

Hmm, no linux/arm64 version of 0.18.5 appears to have been published to Dockerhub: https://hub.docker.com/r/dessalines/lemmy/tags

[dessalines]

@TheSilkky would you be able to help out with #3972