Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move the GC checksum from labels to annotations #362

Merged
merged 1 commit into from
Jun 14, 2021
Merged

Move the GC checksum from labels to annotations #362

merged 1 commit into from
Jun 14, 2021

Conversation

JaneLiuL
Copy link
Contributor

@JaneLiuL JaneLiuL commented Jun 9, 2021 •
edited by stefanprodan
Loading

This PR moves the kustomize.toolkit.fluxcd.io/checksum from labels to annotations. This should fix any conflicts with 3rd party controllers like Stash (fix: #315) that are copying the labels from their custom resources to generated objects.

@JaneLiuL JaneLiuL changed the title detect changes base on diff and move checksum into annotation Draft: detect changes base on diff and move checksum into annotation Jun 9, 2021
stefanprodan
stefanprodan reviewed Jun 10, 2021
View reviewed changes
controllers/utils.go Outdated Show resolved Hide resolved
controllers/kustomization_gc.go Show resolved Hide resolved
@stefanprodan stefanprodan mentioned this pull request Jun 10, 2021
Closed
@stefanprodan
Copy link
Member

@JaneLiuL I found some serious issues with diff, for example, if a user adds namespaces and other objects in those namespaces, then the diff fails while the apply works. I see no way around this limitation but to detect diff failures, ignore them and switch back to the old way of detecting changes.

@stefanprodan stefanprodan mentioned this pull request Jun 11, 2021
Closed
@JaneLiuL JaneLiuL changed the title Draft: detect changes base on diff and move checksum into annotation Move the GC checksum from labels to annotations Jun 11, 2021
stefanprodan
stefanprodan reviewed Jun 11, 2021
View reviewed changes
controllers/utils.go Outdated
// --- /tmp/LIVE-656588208/kustomize.toolkit.fluxcd.io.v1beta1.Kustomization.namespace.name 2021-06-07 12:58:20.738794982 +0200
// +++ /tmp/MERGED-532750671/kustomize.toolkit.fluxcd.io.v1beta1.Kustomization.namespace.name 2021-06-07 12:58:20.798795908 +0200
// @@ -0,0 +1,36 @@
func parseDiffOutput(in []byte) map[string]string {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove this as it's unrelated to this PR.

controllers/kustomization_generator.go Outdated
@@ -21,12 +21,10 @@ import (
"crypto/sha1"
"encoding/json"
"fmt"
"github.com/fluxcd/pkg/apis/kustomize"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be added further down, the first import section is for Go std lib.

controllers/kustomization_generator.go Outdated
}
}
if !exists {
//kus.Transformers = append(kus.Transformers, transformerFileName)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove these commented lines.

controllers/kustomization_controller.go Outdated
@@ -648,6 +649,9 @@ func (r *KustomizationReconciler) apply(ctx context.Context, kustomization kusto
}
}

changeSet := ""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is unrelated to this PR please undo it.

stefanprodan
stefanprodan reviewed Jun 11, 2021
View reviewed changes
Copy link
Member

@stefanprodan stefanprodan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the GC docs here https://github.com/fluxcd/kustomize-controller/blob/main/docs/spec/v1beta1/kustomization.md#garbage-collection

controllers/kustomization_gc.go Outdated
@@ -137,9 +137,25 @@ func (kgc *KustomizeGarbageCollector) Prune(timeout time.Duration, name string,
return changeSet, true
}

// We can't drop the label check in this version.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please replace this with:

Check both labels and annotations for the checksum to preserve backwards compatibility

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix done~

stefanprodan
stefanprodan reviewed Jun 11, 2021
View reviewed changes
controllers/kustomization_generator.go
func (kg *KustomizeGenerator) generateLabelTransformer(checksum, dirPath string) error {
labels := selectorLabels(kg.kustomization.GetName(), kg.kustomization.GetNamespace())

// add checksum label only if GC is enabled
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been lost, please restore it for the annotions

controllers/kustomization_generator.go Outdated
return err
}

labelsFile := filepath.Join(dirPath, transformerAnnotationFileName)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
labelsFile := filepath.Join(dirPath, transformerAnnotationFileName)
annotationsFile := filepath.Join(dirPath, transformerAnnotationFileName)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix done~

stefanprodan
stefanprodan reviewed Jun 11, 2021
View reviewed changes
controllers/kustomization_generator.go Outdated
)

const (
transformerFileName = "kustomization-gc-labels.yaml"
transformerFileName = "kustomization-gc-labels.yaml"
transformerAnnotationFileName = "kustomization-annotation-labels.yaml"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
transformerAnnotationFileName = "kustomization-annotation-labels.yaml"
transformerAnnotationFileName = "kustomization-gc-annotations.yaml"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix done~

@JaneLiuL
Move the GC checksum from labels to annotations
6979dba 
Signed-off-by: Jane Liu L <jane.l.liu@ericsson.com>
stefanprodan
stefanprodan approved these changes Jun 14, 2021
View reviewed changes
Copy link
Member

@stefanprodan stefanprodan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thanks @JaneLiuL 🌷

PS. I'll merge this and do a followup PR to update the GC documentation.

@stefanprodan stefanprodan added area/kustomize Kustomize related issues and pull requests enhancement New feature or request labels Jun 14, 2021
@stefanprodan stefanprodan merged commit 2838997 into fluxcd:main Jun 14, 2021
This was referenced Jun 14, 2021
Merged
Merged
@stefanprodan stefanprodan mentioned this pull request Aug 18, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stefanprodan stefanprodan stefanprodan approved these changes

Assignees
No one assigned
Labels
area/kustomize Kustomize related issues and pull requests enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Garbage collection conflicts with Stash controller
2 participants
@JaneLiuL @stefanprodan