Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mcr.microsoft.com hanging: pulling images failed #165

Open
lucasassisrosa opened this issue Jul 23, 2024 · 18 comments
Open

mcr.microsoft.com hanging: pulling images failed #165

lucasassisrosa opened this issue Jul 23, 2024 · 18 comments
Assignees

Comments

@lucasassisrosa
Copy link

Hi there,

we're having issues since yesterday on pulling images from mcr.microsoft.com. It's hanging forever or failing with connection reset.

Image we're trying to pull now is: mcr.microsoft.com/playwright:v1.37.0-focal@sha256:76de21fcd165e867021bc510c7cdf1811b68b518ba11a2101c3ec7d737eed831. We also tried with a different playwright version.

Issue log:

20:23:31  #8 sha256:d4ba1144e798cf86d9cad1dc30c84fdab2122da75494e92b92b298f10276b224 83.89MB / 568.48MB 235.7s
20:23:57  #8 sha256:d4ba1144e798cf86d9cad1dc30c84fdab2122da75494e92b92b298f10276b224 0B / 568.48MB 261.0s
20:24:02  #8 sha256:d4ba1144e798cf86d9cad1dc30c84fdab2122da75494e92b92b298f10276b224 14.68MB / 568.48MB 266.0s
20:24:07  #8 sha256:d4ba1144e798cf86d9cad1dc30c84fdab2122da75494e92b92b298f10276b224 14.68MB / 568.48MB 271.2s
20:24:12  #8 sha256:d4ba1144e798cf86d9cad1dc30c84fdab2122da75494e92b92b298f10276b224 14.68MB / 568.48MB 276.2s
20:24:17  #8 sha256:d4ba1144e798cf86d9cad1dc30c84fdab2122da75494e92b92b298f10276b224 14.68MB / 568.48MB 281.4s
20:24:22  #8 sha256:d4ba1144e798cf86d9cad1dc30c84fdab2122da75494e92b92b298f10276b224 14.68MB / 568.48MB 286.6s
20:24:25  #8 ERROR: failed to copy: read tcp 10.97.134.229:39880->150.171.69.10:443: read: connection reset by peer
20:24:25  ------
20:24:25   > [build 1/6] FROM mcr.microsoft.com/playwright:v1.37.0-focal@sha256:76de21fcd165e867021bc510c7cdf1811b68b518ba11a2101c3ec7d737eed831:
20:24:25  ------
20:24:25  failed to copy: read tcp 10.97.134.229:39880->150.171.69.10:443: read: connection reset by peer
@bsallesp
Copy link

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 1.08kB 0.0s done
#1 DONE 0.0s

#2 [internal] load metadata for mcr.microsoft.com/dotnet/sdk:8.0
#2 ERROR: failed to do request: Head "https://mcr.microsoft.com/v2/dotnet/sdk/manifests/8.0": EOF

#3 [internal] load metadata for mcr.microsoft.com/dotnet/aspnet:8.0
#3 ERROR: failed to do request: Head "https://mcr.microsoft.com/v2/dotnet/aspnet/manifests/8.0": EOF

[internal] load metadata for mcr.microsoft.com/dotnet/sdk:8.0:



[internal] load metadata for mcr.microsoft.com/dotnet/aspnet:8.0:


Dockerfile:3

1 | #See https://aka.ms/customizecontainer to learn how to customize your debug container and how Visual Studio uses this Dockerfile to build your images for faster debugging.
2 |
3 | >>> FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base
4 | USER root
5 | WORKDIR /app

ERROR: failed to solve: mcr.microsoft.com/dotnet/aspnet:8.0: failed to resolve source metadata for mcr.microsoft.com/dotnet/aspnet:8.0: failed to do request: Head "https://mcr.microsoft.com/v2/dotnet/aspnet/manifests/8.0": EOF

View build details: docker-desktop://dashboard/build/default/default/pgs4yqanfnc4gkq08vsrbq6a2
C:\Users\bsall\source\repos\Quiztle\Quiztle.API\Dockerfile(0,0): Error CTC1014: Docker command failed with exit code 1.
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 1.08kB 0.0s done
#1 DONE 0.0s

#2 [internal] load metadata for mcr.microsoft.com/dotnet/sdk:8.0
#2 ERROR: failed to do request: Head "https://mcr.microsoft.com/v2/dotnet/sdk/manifests/8.0": EOF

#3 [internal] load metadata for mcr.microsoft.com/dotnet/aspnet:8.0
#3 ERROR: failed to do request: Head "https://mcr.microsoft.com/v2/dotnet/aspnet/manifests/8.0": EOF

thats my case

@AndreHamilton-MSFT
Copy link
Contributor

Sorry for the delay. For those facing issues is it intermittent or consistent? Can you run the following

@lucasassisrosa:

curl -A github165 -v mcr.microsoft.com/v2/playwright:v1.37.0-focal/manifests/sha256:76de21fcd165e867021bc510c7cdf1811b68b518ba11a2101c3ec7d737eed831

@bsallesp

curl -A github165 -v https://mcr.microsoft.com/v2/dotnet/sdk/manifests/8.0

In the output of the curl command succeeded you should see an x-msedge-reg value. If you can share that value it would be great. This will help us narrow down where this issue is occurring. Also run it a few times so we have a few values to look into.

image

@AndreHamilton-MSFT AndreHamilton-MSFT self-assigned this Jul 29, 2024
@adanbrownpaca
Copy link

Same issue here guys, any updates? Running these curl commands result in:

  • Host mcr.microsoft.com:443 was resolved.
  • IPv6: 2603:1061:f:101::10, 2603:1061:f:100::10
  • IPv4: 150.171.70.10, 150.171.69.10
  • Trying [2603:1061:f:101::10]:443...
  • Connected to mcr.microsoft.com (2603:1061:f:101::10) port 443
  • schannel: disabled automatic use of client certificate
  • ALPN: curl offers http/1.1
  • Send failure: Connection was reset
  • schannel: failed to send initial handshake data: sent -1 of 292 bytes
  • Closing connection
  • schannel: shutting down SSL/TLS connection with mcr.microsoft.com port 443
  • Send failure: Connection was reset
  • schannel: failed to send close msg: Failed sending data to the peer (bytes written: -1)
    curl: (35) Send failure: Connection was reset

@bsallesp
Copy link

bsallesp commented Aug 1, 2024

Sorry for late, guys.

I forgot to say that, downgrading the docker, it worked.
I am using the version below:

image

@bsallesp
Copy link

bsallesp commented Aug 1, 2024

Looks like a docker desktop issue.

@bsallesp
Copy link

bsallesp commented Aug 1, 2024

Sorry for the delay. For those facing issues is it intermittent or consistent? Can you run the following

@lucasassisrosa:

curl -A github165 -v mcr.microsoft.com/v2/playwright:v1.37.0-focal/manifests/sha256:76de21fcd165e867021bc510c7cdf1811b68b518ba11a2101c3ec7d737eed831

@bsallesp

curl -A github165 -v https://mcr.microsoft.com/v2/dotnet/sdk/manifests/8.0

In the output of the curl command succeeded you should see an x-msedge-reg value. If you can share that value it would be great. This will help us narrow down where this issue is occurring. Also run it a few times so we have a few values to look into.

image

image

@adanbrownpaca
Copy link

It is consistent. I will downgrade my version to 4.32 to confirm that it is a docker desktop issue. Stand by.

@adanbrownpaca
Copy link

Confirmed. Everything is stable using Docker Desktop version 4.32. Thanks - I will follow this thread for further updates and stay on 4.32 in the mean time.

@Matheus-SS
Copy link

I'm having the same problem right now building my images using jenkins and a linux machine. The only solution for me was to download the images before hand and build after that.
Im using mcr.microsoft.com/dotnet/aspnet:7.0 and mcr.microsoft.com/dotnet/sdk:7.0

@K-Benton-CAVC
Copy link

Has anyone resolved this other than by downgrading? The same issue is occurring consistently with podman. Running the curl command as requested seems successful (I get < X-MSEdge-Ref: Ref A: 9F14B8D7F3A74CFAAE7B50947911C634 Ref B: EWR311000107049 Ref C: 2024-09-11T14:39:47Z and a list of checksums, etc.) but when running a build, it's all I/O timeout errors. Very confusing and frustrating.

@valentin-pouget-forge
Copy link

I downgraded to 4.32MacOS Silicon too, I was experiencing the following issues.

broken build because of a network issue only when OpenVPN was enabled:
failed to solve: node:20.16.0-alpine: failed to resolve source metadata for docker.io/library/node:20.16.0-alpine: failed to do request: Head "https://registry-1.docker.io/v2/library/node/manifests/20.16.0-alpine": dialing registry-1.docker.io:443 container via direct connection because app settings has no HTTPS proxy: connecting to registry-1.docker.io:443: dial tcp: lookup registry-1.docker.io: no such host

connecting to an API endpoint from inside a running container was broken, and timeouting during the authentication endpoint. discovery from inside a running container:
Error in callback: RPError: outgoing request timed out after 3500ms

Authentication issues to pull images
failed to solve: node:20.16-alpine: failed to resolve source metadata for [docker.io/library/node:20.16-alpine:](http://docker.io/library/node:20.16-alpine:) failed to authorize: failed to fetch anonymous token: Get "https://auth.docker.io/token?scope=repository%3Alibrary%2Fnode%3Apull&service=registry.docker.io": dial tcp: lookup [auth.docker.io](http://auth.docker.io/) on 127.0.0.11:53: server misbehaving

@ggrafhalfbrick
Copy link

Any news on this without downgrading Docker?

@peereflits
Copy link

peereflits commented Oct 23, 2024

Failing to pull docker images from mcr.microsoft.com solved by turning off IPv6 on Win11 pro 23H2

The error I got, was:

failed to resolve source metadata for mcr.microsoft.com/dotnet/sdk:8.0: failed to do request: Head "https://mcr.microsoft.com/v2/dotnet/sdk/manifests/8.0": EOF

After some investigation and testing I was able to solve this issue by disabling IPv6.

See: https://github.com/peereflits/Articles/blob/main/Other/2024-10-23-failing-to-pull-docker-images.md

@ggrafhalfbrick
Copy link

I ended up having to downgrade to 4.32.0 (157355). That works fine for us.
https://docs.docker.com/desktop/release-notes/

@cseuzaron
Copy link

Downgrade solved my problem as well, thank you!

@jabacrack
Copy link

jabacrack commented Nov 19, 2024

I try to downgrade to 4.32.0 and switch off IPv6 - doesn't help.
Windows 10 Pro 22H2

@cesar-albusac
Copy link

Upgrading Docker Desktop (to 4.36.0) solved my issue

@jabacrack
Copy link

The problem was with my antivirus, after reinstalling the antivirus the docker started working.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests