Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Flux Kustomize #12139

Closed
anthr76 opened this issue Oct 13, 2021 · 12 comments · Fixed by #29224
Closed

Support Flux Kustomize #12139

anthr76 opened this issue Oct 13, 2021 · 12 comments · Fixed by #29224
Assignees
Labels
manager:flux Flux manager priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others status:in-progress Someone is working on implementation type:feature Feature (new functionality)

Comments

@anthr76
Copy link

anthr76 commented Oct 13, 2021

What would you like Renovate to be able to do?

Support Flux v2 kustomize https://fluxcd.io/docs/components/kustomize/api/ kustomize.toolkit.fluxcd.io/v1beta2

If you have any ideas on how this should be implemented, please tell us here.

Take existing ideas and code for native Kustomize and port for usability with the native Flux kustomize spec. Flux maintainers metion remote-urls within native Kustomize are expected not to work well with Flux.

Having renovate track native Flux kustomizes helps track CRDs and keep them up to date.

An example of usage:

https://github.com/onedr0p/home-cluster/blob/7c9106095012e69f4fc1d3e30d41b9ff8f1ce4d8/.github/renovate.json5#L72-L81

https://github.com/onedr0p/home-cluster/blob/7c9106095012e69f4fc1d3e30d41b9ff8f1ce4d8/cluster/crds/rook-ceph/crds.yaml#L11-L12

Is this a feature you are interested in implementing yourself?

No

@anthr76 anthr76 added priority-5-triage status:requirements Full requirements are not yet known, so implementation should not be started type:feature Feature (new functionality) labels Oct 13, 2021
@rarkins rarkins added help wanted Help is needed or welcomed on this issue manager:kustomize Kustomize manager labels Oct 13, 2021
@rarkins
Copy link
Collaborator

rarkins commented Oct 13, 2021

I don't understand what's needed based on the current description so this will probably need to be a community contribution

@anthr76
Copy link
Author

anthr76 commented Oct 13, 2021

Renovate currentley suppports Kustomize (kustomize.config.k8s.io/v1beta1). https://github.com/renovatebot/renovate/blob/main/lib/manager/kustomize/__fixtures__/kustomizeWithLocal.yaml .

Renovate does not support a kustomize.toolkit.fluxcd.io/v1beta2 spec. The flux kustomize spec is a CRD from the GitOps tool Flux v2.

https://github.com/fluxcd/flux2 using remote bases in native Kustomize within Flux doesn't work. So being able to supporrt their CRD spec would be much helpful..

@rarkins rarkins added reproduction:provided auto:reproduction A minimal reproduction is necessary to proceed and removed reproduction:provided labels Oct 13, 2021
@github-actions
Copy link
Contributor

Hi there,

Help us by making a minimal reproduction repository.

Before we can start work on your issue we first need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction to understand what is needed.

We may close the issue if you (or someone else) have not provided a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

@rarkins
Copy link
Collaborator

rarkins commented Oct 13, 2021

Which one of the following is it?

  • New file format (in which case we might treat it as a new manager)
  • Same/similar file format, but syntax inside it is different (treat it as same manager, support both approaches in parallel)

And do the "datasources" it references change? Or are the the same type as before?

Best if we also have a true reproduction repo instead of just an example.

@anthr76
Copy link
Author

anthr76 commented Oct 14, 2021

Hi @rarkins I would consider the flux Kustomization

Same/similar file format, but syntax inside it is different (treat it as same manager, support both approaches in parallel)

The datasource it would reference would be same. Extracting it would be different.

I will include a repoduction repo this weekend.

@gberche-orange
Copy link
Contributor

gberche-orange commented Aug 29, 2022

many thanks to @danports for #13566 which addressed the HelmRelease construct (or should we say "Manager") linked to the helm datasource.

Should the current flux manager be instead split into multiple managers (similarly to the current multiple helm managers: helm-requirements, helm-values, helmfile), such as:

Actually, I wonder how much the flux-system manager adds over the existing kubernetes manager applied on a flux K8S deployment ?

Besides, a new flux-kustomization manager would support the flux KustomizationController CR that reads Kustomization files over the source controller data sources matching renovate datasources:

Flux Source controller CR Renovate DataSources
GitRepository git-tags, git-refs
OCIRepository docker
Bucket TBD aws-s3

@viceice
Copy link
Member

viceice commented Aug 29, 2022

i think this should stay in a single manager

@danports
Copy link
Contributor

I agree with @viceice from both a conceptual and implementation perspective: it makes sense to me to keep all of the Flux code in one manager, and a separate flux-kustomization manager would have to reimplement some of the logic in the flux manager for parsing manifests - would be easier IMO just to extend the flux manager's FluxResource class hierarchy with some additional Kustomization types. I haven't done that myself because I don't use Kustomization resources with external sources very heavily in my Flux deployments - if you want to have a go at it, please feel free! I think you've got a good summary there of the different datasources that could be extracted for the Kustomization resources.

I'm not sure that the kubernetes manager as it stands could fully replace the flux manager's renovation of Flux system manifests, since the kubernetes manager can only update Docker dependencies extracted from Kubernetes manifests, and the flux manager updates the entire Flux system manifest, not just the Docker dependencies contained therein.

@gberche-orange
Copy link
Contributor

thanks @viceice and @danports for your answer and additional details. I now better understand flux system updates, and implications of splitting. I do agree a single flux manager makes sense, sorry for the noise.

As a user the different roles of the manager were hard to grasp for me. I gave a try with simple formatting enhancements and few additional links in #17507

I'll see next if I'd able to contribute flux-kustomization support (both from skill and time allocation perspective).

@gberche-orange
Copy link
Contributor

@rarkins any chance to add label manager:flux so to make this issue appear on the https://docs.renovatebot.com/modules/manager/flux/ page (from what I inferred reading manager.ts) ?

const manager = item.labels
.find((l) => l.name.startsWith('manager:'))
?.name.split(':')[1];
if (!manager) {
continue;
}

@rarkins rarkins added manager:flux Flux manager priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others status:ready and removed priority-5-triage status:requirements Full requirements are not yet known, so implementation should not be started labels Aug 30, 2022
@viceice viceice removed the manager:kustomize Kustomize manager label Aug 30, 2022
@rarkins rarkins added status:requirements Full requirements are not yet known, so implementation should not be started and removed status:ready labels Apr 21, 2023
@rarkins rarkins removed the auto:reproduction A minimal reproduction is necessary to proceed label Oct 1, 2023
@rarkins rarkins added the auto:reproduction A minimal reproduction is necessary to proceed label Oct 1, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Oct 1, 2023

Hi there,

Get your issue fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction.

We may close the issue if you, or someone else, haven't created a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

@rarkins rarkins added status:blocked Issue is blocked by another issue or external requirement and removed status:requirements Full requirements are not yet known, so implementation should not be started labels Oct 1, 2023
@viceice viceice self-assigned this May 23, 2024
@viceice viceice added status:in-progress Someone is working on implementation and removed status:blocked Issue is blocked by another issue or external requirement help wanted Help is needed or welcomed on this issue auto:reproduction A minimal reproduction is necessary to proceed labels May 23, 2024
@renovate-release
Copy link
Collaborator

🎉 This issue has been resolved in version 37.377.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
manager:flux Flux manager priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others status:in-progress Someone is working on implementation type:feature Feature (new functionality)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants