Skip to content

Commit

Permalink
Squashed 'release-tools/' changes from 3b6d17b..5489de6
Browse files Browse the repository at this point in the history
5489de6 Merge pull request kubernetes-sigs#174 from mauriciopoppe/bump-kind-version
0c675d4 Bump kind version to v0.11.1
ef69a88 Merge pull request kubernetes-sigs#173 from nick5616/add-ws2022
44c710c added WS2022 to build platforms
0883be4 Merge pull request kubernetes-sigs#171 from pohly/example-commands
02cda51 build.make: support binaries outside of cmd, with optional go.mod
65922ea Merge pull request kubernetes-sigs#170 from pohly/canary-snapshot-controller
c0bdfb3 prow.sh: deploy canary snapshot-controller in canary jobs
0438f15 Merge pull request kubernetes-sigs#167 from c0va23/feature/release-armv7-image
4786f4d Merge pull request kubernetes-sigs#168 from msau42/update-release-prereq
6a2dc64 Remove requirement to be top-level approver. Only maintainers membership is required to do a release
30a4f7b Release armv7 image
ac8108f Merge pull request kubernetes-sigs#165 from consideRatio/pr/update-github-links-ref-to-master-to-HEAD
999b483 docs: make github links reference HEAD instead of main
fd67069 docs: make github links reference HEAD instead of master
c0a4fb1 Merge pull request kubernetes-sigs#164 from anubha-v-ardhan/patch-1
9c6a6c0 Master to main cleanup
682c686 Merge pull request kubernetes-sigs#162 from pohly/pod-name-via-shell-command
36a29f5 Merge pull request kubernetes-sigs#163 from pohly/remove-bazel
68e43ca prow.sh: remove Bazel build support
c5f59c5 prow.sh: allow shell commands in CSI_PROW_SANITY_POD
71c810a Merge pull request kubernetes-sigs#161 from pohly/mock-test-fixes
9e438f8 prow.sh: fix mock testing
d7146c7 Merge pull request kubernetes-sigs#160 from pohly/kind-update
4b6aa60 prow.sh: update to KinD v0.11.0
7cdc76f Merge pull request kubernetes-sigs#159 from pohly/fix-deployment-selection
ef8bd33 prow.sh: more flexible CSI_PROW_DEPLOYMENT, part II
204bc89 Merge pull request kubernetes-sigs#158 from pohly/fix-deployment-selection
61538bb prow.sh: more flexible CSI_PROW_DEPLOYMENT
2b0e6db Merge pull request kubernetes-sigs#157 from humblec/csi-release
a2fcd6d Adding myself to csi reviewers group
f325590 Merge pull request kubernetes-sigs#149 from pohly/cluster-logs
4b03b30 Merge pull request kubernetes-sigs#155 from pohly/owners
a6453c8 owners: introduce aliases
ad83def Merge pull request kubernetes-sigs#153 from pohly/fix-image-builds
5561780 build.make: fix image publishng
29bd39b Merge pull request kubernetes-sigs#152 from pohly/bump-csi-test
bc42793 prow.sh: use csi-test v4.2.0
b546baa Merge pull request kubernetes-sigs#150 from mauriciopoppe/windows-multiarch-args
bfbb6f3 add parameter base_image and addon_image to BUILD_PARAMETERS
2d61d3b Merge pull request kubernetes-sigs#151 from humblec/cm
48e71f0 Replace `which` command ( non standard)  with `command -v` builtin
feb20e2 prow.sh: collect cluster logs
7b96bea Merge pull request kubernetes-sigs#148 from dobsonj/add-checkpathcmd-to-prow
2d2e03b prow.sh: enable -csi.checkpathcmd option in csi-sanity
09d4151 Merge pull request kubernetes-sigs#147 from pohly/mock-testing
74cfbc9 prow.sh: support mock tests
4a3f110 prow.sh: remove obsolete test suppression
6616a6b Merge pull request kubernetes-sigs#146 from pohly/kubernetes-1.21
510fb0f prow.sh: support Kubernetes 1.21
c63c61b prow.sh: add CSI_PROW_DEPLOYMENT_SUFFIX
51ac11c Merge pull request kubernetes-sigs#144 from pohly/pull-jobs
dd54c92 pull-test.sh: test importing csi-release-tools into other repo
7d2643a Merge pull request kubernetes-sigs#143 from pohly/path-setup
6880b0c prow.sh: avoid creating paths unless really running tests
bc0504a Merge pull request kubernetes-sigs#140 from jsafrane/remove-unused-k8s-libs
5b1de1a go-get-kubernetes.sh: remove unused k8s libs
49b4269 Merge pull request kubernetes-sigs#120 from pohly/add-kubernetes-release
a1e1127 Merge pull request kubernetes-sigs#139 from pohly/kind-for-kubernetes-latest
1c0fb09 prow.sh: use KinD main for latest Kubernetes
1d77cfc Merge pull request kubernetes-sigs#138 from pohly/kind-update-0.10
bff2fb7 prow.sh: KinD 0.10.0
95eac33 Merge pull request kubernetes-sigs#137 from pohly/fix-go-version-check
437e431 verify-go-version.sh: fix check after removal of travis.yml
1748b16 Merge pull request kubernetes-sigs#136 from pohly/go-1.16
ec844ea remove travis.yml, Go 1.16
df76aba Merge pull request kubernetes-sigs#134 from andyzhangx/add-build-arg
e314a56 add build-arg ARCH for building multi-arch images, e.g. ARG ARCH FROM k8s.gcr.io/build-image/debian-base-${ARCH}:v2.1.3
7bc70e5 Merge pull request kubernetes-sigs#129 from pohly/squash-documentation
e0b02e7 README.md: document usage of --squash
316cb95 Merge pull request kubernetes-sigs#132 from yiyang5055/bugfix/boilerplate
26e2ab1 fix: default boilerplate path
1add8c1 Merge pull request kubernetes-sigs#133 from pohly/kubernetes-1.20-tag
3e811d6 prow.sh: fix "on-master" prow jobs
1d60e77 Merge pull request kubernetes-sigs#131 from pohly/kubernetes-1.20-tag
9f10459 prow.sh: support building Kubernetes for a specific version
f7e7ee4 docs: steps for adding testing against new Kubernetes release
fe1f284 Merge pull request kubernetes-sigs#121 from kvaps/namespace-check
8fdf0f7 Merge pull request kubernetes-sigs#128 from fengzixu/master
1c94220 fix: fix a bug of csi-sanity
a4c41e6 Merge pull request kubernetes-sigs#127 from pohly/fix-boilerplate
ece0f50 check namespace for snapshot-controller
dbd8967 verify-boilerplate.sh: fix path to script
9289fd1 Merge pull request kubernetes-sigs#125 from sachinkumarsingh092/optional-spelling-boilerplate-checks
ad29307 Make the spelling and boilerplate checks optional
5f06d02 Merge pull request kubernetes-sigs#124 from sachinkumarsingh092/fix-spellcheck-boilerplate-tests
48186eb Fix spelling and boilerplate errors
71690af Merge pull request kubernetes-sigs#122 from sachinkumarsingh092/include-spellcheck-boilerplate-tests
981be3f Adding spelling and boilerplate checks.
2bb7525 Merge pull request kubernetes-sigs#117 from fengzixu/master
4ab8b15 use the tag to replace commit of csi-test
5d74e45 change the csi-test import path to v4
7dcd0a9 upgrade csi-test to v4.0.2

git-subtree-dir: release-tools
git-subtree-split: 5489de6
  • Loading branch information
spiffxp committed Sep 20, 2021
1 parent 3b6d17b commit a17045a
Show file tree
Hide file tree
Showing 26 changed files with 882 additions and 187 deletions.
16 changes: 16 additions & 0 deletions .prow.sh
Original file line number Diff line number Diff line change
@@ -1,7 +1,23 @@
#! /bin/bash -e

# Copyright 2021 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This is for testing csi-release-tools itself in Prow. All other
# repos use prow.sh for that, but as csi-release-tools isn't a normal
# repo with some Go code in it, it has a custom Prow test script.

./verify-shellcheck.sh "$(pwd)"
./verify-spelling.sh "$(pwd)"
./verify-boilerplate.sh "$(pwd)"
45 changes: 45 additions & 0 deletions KUBERNETES_CSI_OWNERS_ALIASES
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# See the OWNERS docs: https://git.k8s.io/community/contributors/guide/owners.md

aliases:

# SIG-Storage chairs and leads should always have approval rights in all repos.
# Others may be added as needed here or in each repo.
kubernetes-csi-approvers:
- jsafrane
- msau42
- saad-ali
- xing-yang

# Reviewers are automatically assigned to new PRs. The following
# reviewers will be active in all repos. Other reviewers can be
# added in each repo.
#
# Reviewers are encouraged to set the "Busy" flag in their GitHub status
# when they are temporarily unable to review PRs.
kubernetes-csi-reviewers:
- andyzhangx
- chrishenzie
- ggriffiths
- gnufied
- humblec
- j-griffith
- Jiawei0227
- jingxu97
- jsafrane
- pohly
- xing-yang

# This documents who previously contributed to Kubernetes-CSI
# as approver.
emeritus_approver:
- lpabon
- sbezverk
- vladimirvivien

# This documents who previously contributed to Kubernetes-CSI
# as reviewer.
emeritus_reviewer:
- lpabon
- saad-ali
- sbezverk
- vladimirvivien
7 changes: 2 additions & 5 deletions OWNERS
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
# See the OWNERS docs: https://git.k8s.io/community/contributors/guide/owners.md

approvers:
- saad-ali
- msau42
- kubernetes-csi-approvers
- pohly

reviewers:
- saad-ali
- msau42
- pohly
- kubernetes-csi-reviewers
1 change: 1 addition & 0 deletions OWNERS_ALIASES
21 changes: 16 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,11 @@ The expected repository layout is:
Dockerfile in the root when only building a single command
- `Makefile` - includes `release-tools/build.make` and sets
configuration variables
- `.travis.yml` - a symlink to `release-tools/.travis.yml`
- `.prow.sh` script which imports `release-tools/prow.sh`
and may contain further customization
- `.cloudbuild.sh` and `cloudbuild.yaml` as symlinks to
the corresponding files in `release-tools` or (if necessary)
as custom files

To create a release, tag a certain revision with a name that
starts with `v`, for example `v1.0.0`, then `make push`
Expand All @@ -38,16 +42,23 @@ images. Building from master creates the main `canary` image.
Sharing and updating
--------------------

[`git subtree`](https://github.com/git/git/blob/master/contrib/subtree/git-subtree.txt)
[`git subtree`](https://github.com/git/git/blob/HEAD/contrib/subtree/git-subtree.txt)
is the recommended way of maintaining a copy of the rules inside the
`release-tools` directory of a project. This way, it is possible to make
changes also locally, test them and then push them back to the shared
repository at a later time.

We no longer care about importing the full commit history, so `--squash` should be used
when submitting a `release-tools` update. Also make sure that the PR for that
contains the automatically generated commit message in the PR description.
It contains the list of individual commits that were squashed. The script from
https://github.com/kubernetes-csi/csi-release-tools/issues/7 can create such
PRs automatically.

Cheat sheet:

- `git subtree add --prefix=release-tools https://github.com/kubernetes-csi/csi-release-tools.git master` - add release tools to a repo which does not have them yet (only once)
- `git subtree pull --prefix=release-tools https://github.com/kubernetes-csi/csi-release-tools.git master` - update local copy to latest upstream (whenever upstream changes)
- `git subtree add --squash --prefix=release-tools https://github.com/kubernetes-csi/csi-release-tools.git master` - add release tools to a repo which does not have them yet (only once)
- `git subtree pull --squash --prefix=release-tools https://github.com/kubernetes-csi/csi-release-tools.git master` - update local copy to latest upstream (whenever upstream changes)
- edit, `git commit`, `git subtree push --prefix=release-tools git@github.com:<user>/csi-release-tools.git <my-new-or-existing-branch>` - push to a new branch before submitting a PR

verify-shellcheck.sh
Expand Down Expand Up @@ -78,7 +89,7 @@ main

All Kubernetes-CSI repos are expected to switch to Prow. For details
on what is enabled in Prow, see
https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes-csi
https://github.com/kubernetes/test-infra/tree/HEAD/config/jobs/kubernetes-csi

Test results for periodic jobs are visible in
https://testgrid.k8s.io/sig-storage-csi-ci
Expand Down
2 changes: 1 addition & 1 deletion SECURITY_CONTACTS
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# to for triaging and handling of incoming issues.
#
# The below names agree to abide by the
# [Embargo Policy](https://github.com/kubernetes/sig-release/blob/master/security-release-process-documentation/security-release-process.md#embargo-policy)
# [Embargo Policy](https://github.com/kubernetes/sig-release/blob/HEAD/security-release-process-documentation/security-release-process.md#embargo-policy)
# and will be removed and replaced if they violate that agreement.
#
# DO NOT REPORT SECURITY VULNERABILITIES DIRECTLY TO THESE NAMES, FOLLOW THE
Expand Down
69 changes: 54 additions & 15 deletions SIDECAR_RELEASE_PROCESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,8 @@ The release manager must:
* Be a member of the kubernetes-csi organization. Open an
[issue](https://github.com/kubernetes/org/issues/new?assignees=&labels=area%2Fgithub-membership&template=membership.md&title=REQUEST%3A+New+membership+for+%3Cyour-GH-handle%3E) in
kubernetes/org to request membership
* Be a top level approver for the repository. To become a top level approver,
the candidate must demonstrate ownership and deep knowledge of the repository
through active maintainence, responding to and fixing issues, reviewing PRs,
test triage.
* Be part of the maintainers or admin group for the repository. admin is a
superset of maintainers, only maintainers level is required for cutting a
release. Membership can be requested by submitting a PR to kubernetes/org.
* Be part of the maintainers group for the repository.
Membership can be requested by submitting a PR to kubernetes/org.
[Example](https://github.com/kubernetes/org/pull/1467)

## Updating CI Jobs
Expand All @@ -31,16 +26,16 @@ naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.
1. "-on-master" jobs are the closest reflection to the new Kubernetes version.
1. Fixes to our prow.sh CI script can be tested in the [CSI hostpath
repo](https://github.com/kubernetes-csi/csi-driver-host-path) by modifying
[prow.sh](https://github.com/kubernetes-csi/csi-driver-host-path/blob/master/release-tools/prow.sh)
[prow.sh](https://github.com/kubernetes-csi/csi-driver-host-path/blob/HEAD/release-tools/prow.sh)
along with any overrides in
[.prow.sh](https://github.com/kubernetes-csi/csi-driver-host-path/blob/master/.prow.sh)
[.prow.sh](https://github.com/kubernetes-csi/csi-driver-host-path/blob/HEAD/.prow.sh)
to mirror the failing environment. Once e2e tests are passing (verify-unit tests
will fail), then the prow.sh changes can be submitted to [csi-release-tools](https://github.com/kubernetes-csi/csi-release-tools).
1. Changes can then be updated in all the sidecar repos and hostpath driver repo
by following the [update
instructions](https://github.com/kubernetes-csi/csi-release-tools/blob/master/README.md#sharing-and-updating).
instructions](https://github.com/kubernetes-csi/csi-release-tools/blob/HEAD/README.md#sharing-and-updating).
1. New pull and CI jobs are configured by adding new K8s versions to the top of
[gen-jobs.sh](https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes-csi/gen-jobs.sh).
[gen-jobs.sh](https://github.com/kubernetes/test-infra/blob/HEAD/config/jobs/kubernetes-csi/gen-jobs.sh).
New pull jobs that have been unverified should be initially made optional by
setting the new K8s version as
[experimental](https://github.com/kubernetes/test-infra/blob/a1858f46d6014480b130789df58b230a49203a64/config/jobs/kubernetes-csi/gen-jobs.sh#L40).
Expand All @@ -52,7 +47,7 @@ naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.
1. Identify all issues and ongoing PRs that should go into the release, and
drive them to resolution.
1. Download v2.8+ [K8s release notes
generator](https://github.com/kubernetes/release/tree/master/cmd/release-notes)
generator](https://github.com/kubernetes/release/tree/HEAD/cmd/release-notes)
1. Generate release notes for the release. Replace arguments with the relevant
information.
* Clean up old cached information (also needed if you are generating release
Expand Down Expand Up @@ -95,12 +90,56 @@ naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.
1. Check [image build status](https://k8s-testgrid.appspot.com/sig-storage-image-build).
1. Promote images from k8s-staging-sig-storage to k8s.gcr.io/sig-storage. From
the [k8s image
repo](https://github.com/kubernetes/k8s.io/tree/master/k8s.gcr.io/images/k8s-staging-sig-storage),
repo](https://github.com/kubernetes/k8s.io/tree/HEAD/k8s.gcr.io/images/k8s-staging-sig-storage),
run `./generate.sh > images.yaml`, and send a PR with the updated images.
Once merged, the image promoter will copy the images from staging to prod.
1. Update [kubernetes-csi/docs](https://github.com/kubernetes-csi/docs) sidecar
and feature pages with the new released version.
1. After all the sidecars have been released, update
CSI hostpath driver with the new sidecars in the [CSI repo](https://github.com/kubernetes-csi/csi-driver-host-path/tree/master/deploy)
CSI hostpath driver with the new sidecars in the [CSI repo](https://github.com/kubernetes-csi/csi-driver-host-path/tree/HEAD/deploy)
and [k/k
in-tree](https://github.com/kubernetes/kubernetes/tree/master/test/e2e/testing-manifests/storage-csi/hostpath/hostpath)
in-tree](https://github.com/kubernetes/kubernetes/tree/HEAD/test/e2e/testing-manifests/storage-csi/hostpath/hostpath)

## Adding support for a new Kubernetes release

1. Add the new release to `k8s_versions` in
https://github.com/kubernetes/test-infra/blob/090dec5dd535d5f61b7ba52e671a810f5fc13dfd/config/jobs/kubernetes-csi/gen-jobs.sh#L25
to enable generating a job for it. Set `experimental_k8s_version`
in
https://github.com/kubernetes/test-infra/blob/090dec5dd535d5f61b7ba52e671a810f5fc13dfd/config/jobs/kubernetes-csi/gen-jobs.sh#L40
to ensure that the new jobs aren't run for PRs unless explicitly
requested. Generate and submit the new jobs.
1. Create a test PR to try out the new job in some repo with `/test
pull-kubernetes-csi-<repo>-<x.y>-on-kubernetes-<x.y>` where x.y
matches the Kubernetes release. Alternatively, run .prow.sh in that
repo locally with `CSI_PROW_KUBERNETES_VERSION=x.y.z`.
1. Optional: update to a [new
release](https://github.com/kubernetes-sigs/kind/tags) of kind with
pre-built images for the new Kubernetes release. This is optional
if the current version of kind is able to build images for the new
Kubernetes release. However, jobs require less resources when they
don't need to build those images from the Kubernetes source code.
This change needs to be tried out in a PR against a component
first, then get submitted against csi-release-tools.
1. Optional: propagate the updated csi-release-tools to all components
with the script from
https://github.com/kubernetes-csi/csi-release-tools/issues/7#issuecomment-707025402
1. Once it is likely to work in all components, unset
`experimental_k8s_version` and submit the updated jobs.
1. Once all sidecars for the new Kubernetes release are released,
either bump the version number of the images in the existing
[csi-driver-host-path
deployments](https://github.com/kubernetes-csi/csi-driver-host-path/tree/HEAD/deploy)
and/or create a new deployment, depending on what Kubernetes
release an updated sidecar is compatible with. If no new deployment
is needed, then add a symlink to document that there intentionally
isn't a separate deployment. This symlink is not needed for Prow
testing because that will use "kubernetes-latest" as fallback.
Update that link when creating a new deployment.
1. Create a new csi-driver-host-path release.
1. Bump `CSI_PROW_DRIVER_VERSION` in prow.sh to that new release and
(eventually) roll that change out to all repos by updating
`release-tools` in them. This is used when testing manually. The
Prow jobs override that value, so also update
`hostpath_driver_version` in
https://github.com/kubernetes/test-infra/blob/91b04e6af3a40a9bcff25aa030850a4721e2dd2b/config/jobs/kubernetes-csi/gen-jobs.sh#L46-L47
13 changes: 13 additions & 0 deletions boilerplate/boilerplate.Dockerfile.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Copyright YEAR The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
13 changes: 13 additions & 0 deletions boilerplate/boilerplate.Makefile.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Copyright YEAR The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
13 changes: 13 additions & 0 deletions boilerplate/boilerplate.bzl.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Copyright YEAR The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
15 changes: 15 additions & 0 deletions boilerplate/boilerplate.go.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
/*
Copyright YEAR The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
Loading

0 comments on commit a17045a

Please sign in to comment.