Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stop using gs://kubernetes-release bucket, use dl.k8s.io for downloads #14828

Closed
afbjorklund opened this issue Aug 21, 2022 · 9 comments · Fixed by #15840
Closed

Stop using gs://kubernetes-release bucket, use dl.k8s.io for downloads #14828

afbjorklund opened this issue Aug 21, 2022 · 9 comments · Fixed by #15840
Labels
area/build-release kind/improvement Categorizes issue or PR as related to improving upon a current feature. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete.

Comments

@afbjorklund
Copy link
Collaborator

afbjorklund commented Aug 21, 2022 •
edited
Loading

The downloads should be using "dl.k8s.io" instead.

The hardcoded urls might cause a problem in the future
(like is happening today, with the registry and packages)

i.e. no more storage.googleapis.com/kubernetes-release

See:
@afbjorklund
Copy link
Collaborator Author

afbjorklund commented Aug 21, 2022 •
edited
Loading

@medyagh another google infrastructure cleanup item...

https://dl.k8s.io/release/stable.txt -->
https://storage.googleapis.com/kubernetes-release/release/stable.txt

I guess it also affects the "minikube" subdirectory, eventually ?

But primarily it was about kubeadm/kubelet/kubectl downloads:

// DefaultKubeBinariesURL returns a URL to kube binaries
func DefaultKubeBinariesURL() string {
        return fmt.Sprintf("https://%s/kubernetes-release/release", downloadHost)
}

var downloadHost = "storage.googleapis.com" (plus mirror)

The hardcoded bucket subdirectory needs go in the url/host instead.

@afbjorklund
Copy link
Collaborator Author

afbjorklund commented Aug 21, 2022 •
edited
Loading

Current procedure follows upstream: ("Without a package manager")
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl

sudo curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/${ARCH}/{kubeadm,kubelet,kubectl}
sudo chmod +x {kubeadm,kubelet,kubectl}
        base := fmt.Sprintf("%s/%s/bin/%s/%s/%s", binaryURL, version, osName, archName, binaryName)

@afbjorklund
Copy link
Collaborator Author

Now changed to use "dl.k8s.io" (and removed kubectl)

@afbjorklund afbjorklund mentioned this issue Nov 14, 2022
Open
@afbjorklund afbjorklund added the kind/improvement Categorizes issue or PR as related to improving upon a current feature. label Dec 11, 2022
@afbjorklund afbjorklund changed the title Stop using gs://kubernetes-release bucket Stop using gs://kubernetes-release bucket, use dl.k8s.io for downloads Feb 3, 2023
@afbjorklund afbjorklund mentioned this issue Feb 6, 2023
Closed
@sftim
Copy link

sftim commented Feb 13, 2023

We are also keen to reduce our reliance on GCP (as in, on only GCP) due to financial concerns.

@afbjorklund
Copy link
Collaborator Author

afbjorklund commented Feb 13, 2023 •
edited
Loading

Note that:

Most minikube users will get their binaries from the "preload" (GCS), not from the dl.k8s.io

If the Kubernetes project really wanted to make the downloads smaller, it would use .tar.xz ?

45M	kubeadm
46M	kubectl
116M	kubelet
207M	total

14M	kubeadm.tar.xz
15M	kubectl.tar.xz
30M	kubelet.tar.xz
58M	total

Should still do this change. Will make a PR...

Wonder if dl.k8s.io also works in China ?

@sftim
Copy link

sftim commented Feb 13, 2023

The approach we recommend for end users subject to state-level censorship is to use a trusted mirror within their state's allowed boundary (we don't provide such a mirror).

@afbjorklund
Copy link
Collaborator Author

afbjorklund commented Feb 13, 2023 •
edited
Loading

OK, will continue to use the current mirror then.

        // Mirror CN
        AliyunMirror = "registry.cn-hangzhou.aliyuncs.com/google_containers"
...
        // ImageRepositories contains all known image repositories
        ImageRepositories = map[string][]string{
                "global": {""},
                "cn":     {AliyunMirror},
        }
        aliyunMirror = "kubernetes.oss-cn-hangzhou.aliyuncs.com"
        downloadHost = "storage.googleapis.com"

@afbjorklund afbjorklund mentioned this issue Feb 13, 2023
Merged
@afbjorklund
Copy link
Collaborator Author

Seem to be confusing GCR and GCS while typing, too close alphabetically...

Also, it seems that https://dl.k8s.io/stable.txt does not exist (so sed failed)

@afbjorklund afbjorklund added the priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. label Feb 13, 2023
@afbjorklund
Copy link
Collaborator Author

afbjorklund commented Feb 13, 2023 •
edited
Loading

Apparently the directory is still there, my bad. Needs a release/, and --location (to follow the redirection)

sed -e 's|curl -s https://storage.googleapis.com/kubernetes-release/release|curl -sSL https://dl.k8s.io/release|'

$ curl -s https://dl.k8s.io/release/stable.txt
<html>
<head><title>302 Found</title></head>
<body>
<center><h1>302 Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
$ curl -sSL https://dl.k8s.io/release/stable.txt
v1.26.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/build-release kind/improvement Categorizes issue or PR as related to improving upon a current feature. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Download binaries from dl.k8s.io instead of GCS afbjorklund/minikube
2 participants
@afbjorklund @sftim