proxy.golang.org: fails to fetch github.com/gmlewis/go-fonts

[gmlewis]

What is the URL of the page with the issue?

https://pkg.go.dev/github.com/gmlewis/go-fonts@v0.16.0

What is your user agent?

[Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36](https://explore.whatismybrowser.com/useragents/parse/?analyse-my-user-agent=yes)

Screenshot

What did you do?

I visited https://pkg.go.dev/github.com/gmlewis/go-fonts@v0.16.0 multiple times over the course of about 15 minutes, each time asking for an update.
Eventually, that URL no longer said that that version couldn't be found, but instead started redirecting to the page with the old version.

What did you expect to see?

The latest version.

What did you see instead?

The old version.

[gabyhelp]

Similar Issues

• x/pkgsite: fails to update latest to reflect latest version #46985
• x/pkgsite: older version is displayed as "Latest". #63176
• x/pkgsite: #48949
• x/pkgsite: Update go version on docs #53670
• x/pkgsite: go-source meta tag information is not automatically updated if it changes for an existing module version #46203

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[hyangah]

According to https://proxy.golang.org/github.com/gmlewis/go-fonts/@v/list
proxy.golang.org knows only up to v0.0.13. Pkg.go.dev only learns about module versions through proxy.golang.org, so it is working as intended.

From the proxy's log, I see that proxy.golang.org keeps failing on this repo for unknown reasons.

cc @suzmue @golang/tools-team

[adonovan]

Indeed, it seems to be a problem with the origin server.

xtools$ GOPROXY=direct go mod download -x github.com/gmlewis/go-fonts@v0.16.0
mkdir -p /Users/adonovan/go/pkg/mod/cache/vcs # git3 https://github.com/gmlewis/go-fonts
# lock /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51.lock
# /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51 for git3 https://github.com/gmlewis/go-fonts
cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git tag -l
0.016s # cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git tag -l
cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git ls-remote -q origin
0.128s # cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git ls-remote -q origin
cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2 --
0.011s # cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2 --
cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c protocol.version=2 fetch -f --depth=1 origin refs/tags/v0.16.0:refs/tags/v0.16.0
50.136s # cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c protocol.version=2 fetch -f --depth=1 origin refs/tags/v0.16.0:refs/tags/v0.16.0
cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' refs/tags/v0.16.0 --
0.011s # cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' refs/tags/v0.16.0 --
cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
0.010s # cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
0.009s # cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
0.010s # cd /Users/adonovan/go/pkg/mod/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
# get https://sum.golang.org/lookup/github.com/gmlewis/go-fonts@v0.16.0
# get https://sum.golang.org/lookup/github.com/gmlewis/go-fonts@v0.16.0: 500 Internal Server Error (58.136s)
go: github.com/gmlewis/go-fonts@v0.16.0: verifying go.mod: github.com/gmlewis/go-fonts@v0.16.0/go.mod: reading https://sum.golang.org/lookup/github.com/gmlewis/go-fonts@v0.16.0: 500 Internal Server Error

Disabling the sumdb, it runs to completion, albeit slowly:

xtools$ time GOMODCACHE=$(mktemp -d) GOSUMDB=off GOPROXY=direct go mod download -x github.com/gmlewis/go-fonts@v0.16.0
mkdir -p /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs # git3 https://github.com/gmlewis/go-fonts
# lock /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51.lock
mkdir -p /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51 # git3 https://github.com/gmlewis/go-fonts
cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git init --bare
0.028s # cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git init --bare
cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git remote add origin -- https://github.com/gmlewis/go-fonts
0.009s # cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git remote add origin -- https://github.com/gmlewis/go-fonts
cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git tag -l
0.008s # cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git tag -l
cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git ls-remote -q origin
0.152s # cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git ls-remote -q origin
cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2 --
0.013s # cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2 --
cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c protocol.version=2 fetch -f --depth=1 origin refs/tags/v0.16.0:refs/tags/v0.16.0
49.405s # cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c protocol.version=2 fetch -f --depth=1 origin refs/tags/v0.16.0:refs/tags/v0.16.0
cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' refs/tags/v0.16.0 --
0.017s # cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' refs/tags/v0.16.0 --
cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
0.016s # cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
0.011s # cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
0.010s # cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
0.011s # cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
0.012s # cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git cat-file blob 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2:go.mod
cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c core.autocrlf=input -c core.eol=lf archive --format=zip --prefix=prefix/ 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2
34.982s # cd /var/folders/fy/dn6v01n16zjdwsqy_8qfbbxr000_9w/T/tmp.nEs0ebhRxI/cache/vcs/82e0d29a7e1123c37c06e94c9f9310af2c093c0854570f189fbc3358dc25cb51; git -c core.autocrlf=input -c core.eol=lf archive --format=zip --prefix=prefix/ 2c7b6b4f53ba603b21877e400d2b2c60fbcd2ef2

So it seems the upstream git fetch and git archive operations for this repo are so slow that they exceed the timeouts of the sumdb and the proxy. FWIW, the repo contains almost 1GB of font data and very little Go code.

[seankhliao]

is this the 500MB limit for proxy.golang.org?

[adonovan]

Seems like sumdb is the bottlenneck, but it could be for a similar reason. I don't know, but I bet @suzmue does.

[suzmue]

sum.golang.org and proxy.golang.org have the same limits and issues.

It appears that the proxy is not able to even successfully download the module in order to check the 500MB limit, but does seem that this has to do with the size as the fetch uses a lot of memory / takes a long time in the proxy's fetch environment.

[rsc]

I looked into this.

The repo itself is 1.2 GB after git clone --depth=1; 385 MB is the .git directory and 815 MB is the actual content.

The limit on the uncompressed size of an individual module is 500 MB (500<<20 bytes).

I see that there are many modules in this repo and each is small, but we also have to impose some limit on how large a git repo we are willing to download to get a module. It seems reasonable to me to say that this repo is too large. The solution would be to break it up into multiple modules.

Apologies, @gmlewis.

[gmlewis]

Thank you, everyone, who looked into this issue!

I really appreciate your effort and I'm sorry for the enormity of the repo.
I will break it up. That might be a cleaner solution anyway.

Closing with apologies.