Introduce a regular patch release schedule for CA

[MaciekPytel]

No description provided.

[MaciekPytel]

@gjtempleton @towca @x13n I wrote down the proposal I discussed with you earlier on. I assigned release dates randomly, please suggest changes if you plan to be unavailable during your shift.

/hold
To get all the lgtms.

[towca]

/lgtm

[x13n]

/lgtm

That's a bit outside of the scope of what you're adding, but I think that the new section should also include info about which minors actually are in scope for getting patched at all.

[MaciekPytel]

@x13n I think it already does - "Cluster Autoscaler releases patches for versions corresponding to currently
supported Kubernetes versions". The link is updated and contains a list of versions.

[elmiko]

i think this is nice, thanks @MaciekPytel
+1

[gjtempleton]

/lgtm

[aaron-trout]

Can we describe in the process what is in-scope / out-of-scope for patch releases?

I joined the sig-autoscaling channel/call recently to look at regular rebuilds/release for security reasons (vulnerabilities in the existing images) and this seems to go a long way towards fixing that already. I'm wondering if it's reasonable to also update go dependencies on the release branches when creating these patch releases (at least for those with HIGH+ CVEs maybe?)

[MaciekPytel]

@aaron-trout That's a good point. A patch release would always be built using the newest base image, so it should address any vulnerabilities in the images. We could argue if 2-month latency is good enough, but we explicitly say additional releases may happen in response to vulnerabilities and if a serious vulnerability is escalated I'd imagine we'd be open to building a release in response (best escalation channel is probably via sig-meeting, either by participating or just adding to agenda).

Dependencies are more tricky - historically we haven't bumped them on patch releases, unless there was some specific bug that required us to do so. And vast majority of our dependencies are transitive dependencies of kubernetes, which we don't monitor directly. I don't think we want to get into the business of checking all those deps we're not even familiar with for vulns and even less so into solving any conflicts between kubernetes transitive deps and kubernetes itself.
That being said I'd imagine kubernetes itself is much better at managing this than we're (they have a sig dedicated to security after all), so maybe committing to bumping k8s to the latest patch version (which in turn will bump any transitive deps) on any patch release would be a reasonable middle ground here?

@x13n @towca @gjtempleton thoughts?

[x13n]

@x13n I think it already does - "Cluster Autoscaler releases patches for versions corresponding to currently supported Kubernetes versions". The link is updated and contains a list of versions.

Of course I missed the first sentence of the diff 🤦 Thanks, just lgtm from me then!

[x13n]

Dependencies are more tricky - historically we haven't bumped them on patch releases, unless there was some specific bug that required us to do so. And vast majority of our dependencies are transitive dependencies of kubernetes, which we don't monitor directly. I don't think we want to get into the business of checking all those deps we're not even familiar with for vulns and even less so into solving any conflicts between kubernetes transitive deps and kubernetes itself. That being said I'd imagine kubernetes itself is much better at managing this than we're (they have a sig dedicated to security after all), so maybe committing to bumping k8s to the latest patch version (which in turn will bump any transitive deps) on any patch release would be a reasonable middle ground here?

I think bumping k8s deps should be good enough unless someone explicitly reaches out about a critical vulnerability in some non-k8s deps in which case we should also ensure the fix is included.

[aaron-trout]

Sounds good. It may be worth doing this (checking for vulns and updating transitive deps) as a one off when implementing this new release policy since many of the images have HIGH / CRITICAL CVEs currently. For example here is a scan of the 1.24.0 image using trivy:

I'm also happy to do some local builds off the release branches and scan the resulting images for comparison if that is helpful? I can raise some PRs to the release branches to attempt to fix any found issues.

[mwielgus]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: MaciekPytel, mwielgus

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cluster-autoscaler/OWNERS [MaciekPytel,mwielgus]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment