[buildx/arm64] yarn and npm not able to download packages from internet

[voydz]

Hi there,

thanks for your awesome work with this project. I am experiencing issues using buildx for the arm64 arch. Everything is running smooth until yarn or npm is trying to download packages over the network.

This is an snipped from build-log running on GitHub actions while using yarn:

#22 [linux/arm64 build-deps 5/6] RUN yarn --production
#22 176.3 info There appears to be trouble with your network connection. Retrying...
#22 195.3 error An unexpected error occurred: "https://registry.yarnpkg.com/date-fns/-/date-fns-2.16.1.tgz: ESOCKETTIMEDOUT".

Running the same buildx build locally the output is a bit more concise:

 => [linux/arm64 build-deps 5/6] RUN yarn --production
 => => # Unknown QEMU_IFLA_INFO_KIND ipip                                                                                                
 => => # Unknown QEMU_IFLA_INFO_KIND ip6tnl                                                                                              
 => => # yarn install v1.22.5                                                                                                            
 => => # Unknown QEMU_IFLA_INFO_KIND ipip                                                                                                
 => => # Unknown QEMU_IFLA_INFO_KIND ip6tnl                                                                                              
 => => # [1/4] Resolving packages...                                                                                   
 => => # [2/4] Fetching packages...                                                                                                      
 => => # Unknown QEMU_IFLA_INFO_KIND ipip                                                                                                
 => => # Unknown QEMU_IFLA_INFO_KIND ip6tnl                                                                                              
 => => # info There appears to be trouble with your network connection. Retrying...  

The command used to start the build:
docker buildx build --platform linux/amd64,linux/arm64 -t myrepo/myproject:latest --push client/

My Dockerfile looks like this:
(Note that I do not copy node_modules into build context. The files to copy are managed using a .dockerignore file.)

# Stage 1 - the build process
FROM node:latest as build-deps
WORKDIR /usr/src/app

ARG REACT_APP_GRAPH_URL
ENV REACT_APP_GRAPH_URL=$REACT_APP_GRAPH_URL

COPY . ./
COPY .env.example .env

RUN yarn --production
RUN yarn build

# Stage 2 - the deployment
FROM nginx:latest
COPY --from=build-deps /usr/src/app/build /usr/share/nginx/html

COPY docker/default /etc/nginx/conf.d/default.conf

Workaround

As a workaround I currently using a .yarnrc to create an offline cache. I copy a it into the build context and install it with the yarn --offline option. That works for now, but I do not like it very much. https://classic.yarnpkg.com/blog/2016/11/24/offline-mirror/

thank you again and best regards

[LaurentGoderre]

ARe you sure your docker is correctly configured to allow internet access to your containers?

[voydz]

Hi @LaurentGoderre ,

As I wrote, the same holds true for running in GitHub Action environments. I already had this idea, so I verified the issue by running the build on other platforms. Also, in every environment amd64 builds just fine.

What do you think I can further do, to help with this issue?

Thanks and best regards

[voydz]

@PeterDaveHello I noticed you added the yarn label to this issue. I also checked this using npm with the exact same behavior as yarn.

Currently I am trying to run some network diagnostic inside the container. If I stumble upon something useful, I will let you know.

best regards

[goshlanguage]

I think this is fairly reproducible. On my ubuntu machines, any Dockerfile that includes RUN npm .... will fail in this same way when either targeting an arm64 image, or running docker buildx build when the targets include an arm64 target.

I am not able to reproduce however on my macbooks.

[JanJakes]

Let me guess – you're not building on an actual arm64 runner node which means the arm64 build is generated using QEMU. Emulation is slow, therefore there's much higher risk of running into timeouts.

It may be still possible there's an additional problem with the network connection which may be worth debugging (I do see ESOCKETTIMEDOUT in the logs) but I'm not sure if this is related to docker-node in any way.

[goshlanguage]

While this seems plausible, I think it's the common case that projects will build against an emulated environment. Would it not be an improvement to docker-node to perhaps update documentation in some way to call this out?

[Sp4Rx]

I'm having similar problem. I have also started a SO thread

[vl-shopback]

got the same issue on arm instance to build amd64 image with emulator

[mohsenasm]

It also stuck in #14 [dependencies 2/5] RUN yarn config set network-timeout 300000 for node:20-alpine and linux/arm/v7.
Super weird!

[39otrebla]

Same here. Less than a minute locally vs. 8 minutes on average running in a Docker build step on GitHub Actions. I confirm we are building an arm64 image with Docker buildx official action.

Also, no way to get it working without using --network-timeout 100000.

[clowa]

@mohsenasm I agree with you. Look's like this effects node:20 (lts). I have downgraded my dockerfile to node:18-alpine3.19 and yarn install now works again.

[razvanphp]

As per the linked PRs, increase your network timeout and be extremely patient if running under buildx/QEMU via GitHub Actions. :-)

I would have NEVER imagined such a difference in build time on a Xeon V4 server:

amd64

• npm install --> added 1285 packages in 2m
• npm build --> built in 29.61s

arm64 w/ QEMU

• npm install --> added 1285 packages in 28m
• npm build --> built in 21m 59s

This is (or should be) unacceptable for any standards.

One way to approach this is by splitting into build stages, where the npm install and build are done only on native auth and the final image is only created with multi arch --platform.

[cw-sarvesh]

As per the linked PRs, increase your network timeout and be extremely patient if running under buildx/QEMU via GitHub Actions. :-)

I would have NEVER imagined such a difference in build time on a Xeon V4 server:

amd64

• npm install --> added 1285 packages in 2m
• npm build --> built in 29.61s

arm64 w/ QEMU

• npm install --> added 1285 packages in 28m
• npm build --> built in 21m 59s

This is (or should be) unacceptable for any standards.

One way to approach this is by splitting into build stages, where the npm install and build are done only on native auth and the final image is only created with multi arch --platform.

Yes I also faced this issue while building multi platform image. The only solution currently I found in my fit is that you create docker image seperately and create multi platform manifest on anyone machine. With parallel building these two image and then combining them will give you better results.

[64bit]

Thought this was multiplatform build issue but Docker on Rapsberry Pi 4 , native aarch64/arm64, has same problem with node 20.12.0. With node 18,20,0 build never seems to finish or fails. In all these cases I used npm, and it did not work.

But yarn seems to be working with both node:18-alpine3.19 and node:20.12-alpine3.19 ( even though build duration is just longer compared to amd64)

[chadlwilson]

Thought this was multiplatform build issue but Docker on Rapsberry Pi 4 , native aarch64/arm64, has same problem with node 20.12.0. With node 18,20,0 build never seems to finish or fails. In all these cases I used npm, and it did not work.

Just to double-check, are you using the newer 64-bit Raspberry Pi OS (as opposed to the older 32-bit versions)?

[64bit]

Actually, tt was Ubuntu 22.04.3 LTS 64-bit on Raspberry Pi.

[silverwind]

People on node 20 with npm 10 could try downgrading npm to v9 because v10 has a known bug where it opens an excessive amount of network connections, see npm/cli#7072.

[andrii33]

Try to use node:lts-alpine3.20 as base image for NodeJS 20 (current LTS version)

Something like

FROM node:lts-alpine3.20 as build

WORKDIR /usr/app

ARG NODE_AUTH_TOKEN

ENV NODE_AUTH_TOKEN=$NODE_AUTH_TOKEN

ARG NODE_ENV

COPY package.json yarn.lock .npmrc ./

RUN apk update && \
    apk add --no-cache \
        python3 \
        make \
        g++ \
        bash
    
RUN yarn

And github actions buildx setup with platforms that required:

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v3
      with:
        platforms: linux/amd64,linux/arm64

Build for both platforms takes like 2 minutes

[pkit]

Works fine (although pretty slow) for node 20.12, fails miserably with 20.16

[pkit]

Ok, something is broken from node 20.13 and later.
I cannot see anything interesting in the changelog though.

[henryfool91]

Timeout to 100000 kinda helped against the error, but the speed... it took 1 hour to deploy..