Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: Optimize cache by minimizing unnecessary data storage. #1405

Closed
wants to merge 1 commit into from

Conversation

mjshastha
Copy link
Collaborator

@mjshastha mjshastha commented Nov 6, 2024

Updates to controller-runtime and client-go for better performance, bug fixes, and memory optimization.

Stripped managed fields and last-applied-configuration from resources and cache:

  • Implemented object metadata transformation using WithTransform in NewSharedInformerFactoryWithOptions for shared informers. ref: informers package - k8s.io/client-go/informers - Go Packages
  • Updated the controller runtime manager cache to include DefaultTransform, leveraging TransformStripManagedFields() along with logic to remove the last-applied-configuration annotation for all cached objects. ref: cache package - sigs.k8s.io/controller-runtime/pkg/cache - Go Packages
  • These changes ensure that unwanted metadata is removed from objects before they are stored in the cache, optimizing memory usage and improving performance.

Additionally, fixed failing unit test cases.

@mjshastha mjshastha requested a review from deven0t November 6, 2024 13:19
@mjshastha mjshastha force-pushed the SBOptmizations branch 5 times, most recently from 999f3b5 to cd11a43 Compare November 7, 2024 03:50
@mjshastha mjshastha force-pushed the SBOptmizations branch 4 times, most recently from 8d0f906 to 914b2e8 Compare December 3, 2024 09:10
@mjshastha mjshastha force-pushed the SBOptmizations branch 9 times, most recently from cb0f499 to 67e1d0f Compare December 10, 2024 09:11
@mjshastha mjshastha force-pushed the SBOptmizations branch 4 times, most recently from 04de140 to 4d7ff27 Compare December 10, 2024 11:09
…ug fixes, and memory optimization.

Stripped managed fields and last-applied-configuration from resources and cache:

Implemented object metadata transformation using `WithTransform` in `NewSharedInformerFactoryWithOptions` for shared informers. ref: informers package - k8s.io/client-go/informers - Go Packages

Updated the controller runtime manager cache to include `DefaultTransform`, leveraging `TransformStripManagedFields()` along with logic to remove the last-applied-configuration annotation for all cached objects. ref: cache package - sigs.k8s.io/controller-runtime/pkg/cache - Go Packages

These changes ensure that unwanted metadata is removed from objects before they are stored in the cache, optimizing memory usage and improving performance.
@@ -36,34 +35,6 @@ type Inputs struct {
func VulnerabilityScannerBehavior(inputs *Inputs) func() {
return func() {

Context("When unmanaged Pod is created", func() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason to remove this test case

@@ -79,15 +50,15 @@ func VulnerabilityScannerBehavior(inputs *Inputs) func() {

err := inputs.Create(ctx, deploy)
Expect(err).ToNot(HaveOccurred())
Eventually(inputs.HasActiveReplicaSet(inputs.PrimaryNamespace, deploy.Name), inputs.AssertTimeout).Should(BeTrue())
Eventually(inputs.HasActiveReplicaSet(inputs.PrimaryNamespace, deploy.Name), 300*time.Second).Should(BeTrue())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see we are updating AssertTimeout to 5min, so we dont need this change here. Let it come from inputs.Asserttimeout

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will address in the next commit.

@mjshastha mjshastha closed this Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants