Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: avoid heap allocation in applier findPackage #7883

Merged
merged 2 commits into from
Dec 9, 2024

Conversation

jinroh
Copy link
Contributor

@jinroh jinroh commented Nov 6, 2024

Description

The findPackage method in the applier code iterate over []ftypes.Package which is a huge struct. The go compiler generates code that makes heap allocations.

This PR changes findPackage so as to not iterate by value, and avoiding such heap allocations.

Checklist

  • I've read the guidelines for contributing to this repository.
  • I've followed the conventions in the PR title.
  • I've added tests that prove my fix is effective or that my feature works.
  • I've updated the documentation with the relevant information (if needed).
  • I've added usage information (if the PR introduces new options)
  • I've included a "before" and "after" example to the description (if the PR is a user interface change).

@CLAassistant
Copy link

CLAassistant commented Nov 6, 2024

CLA assistant check
All committers have signed the CLA.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@knqyf263
Copy link
Collaborator

knqyf263 commented Nov 6, 2024

We'd love to know how much this change makes a difference. Do you mind sharing the benchmark results?

@jinroh
Copy link
Contributor Author

jinroh commented Nov 7, 2024

It's a bit of a contrived benchmark (worst case / micro benchmark) but it highlights the performance difference:

package applier

import (
	"testing"

	"github.com/aquasecurity/trivy/pkg/fanal/types"
)

func BenchmarkFindPackage(b *testing.B) {
	packages := types.Packages{}

	for i := 0; i < 10000; i++ {
		packages = append(packages, types.Package{
			Name:    "foobar",
			Version: "1.2.3",
			Release: "4.5.6",
		})
	}

	needle := types.Package{
		Name:    "openssl",
		Version: "1.2.3",
		Release: "4.5.6",
	}

	packages = append(packages, needle)

	for i := 0; i < b.N; i++ {
		found := findPackage(needle, packages)
		if found == nil {
			b.Fatal("not found")
		}
	}
}

Before:

goos: darwin
goarch: arm64
pkg: github.com/aquasecurity/trivy/pkg/fanal/applier
cpu: Apple M1 Max
BenchmarkFindPackage-10    	    1086	   1065229 ns/op
PASS
ok  	github.com/aquasecurity/trivy/pkg/fanal/applier	2.701s

After:

goos: darwin
goarch: arm64
pkg: github.com/aquasecurity/trivy/pkg/fanal/applier
cpu: Apple M1 Max
BenchmarkFindPackage-10    	   71274	     16479 ns/op
PASS
ok  	github.com/aquasecurity/trivy/pkg/fanal/applier	2.782s

Copy link
Collaborator

@knqyf263 knqyf263 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@knqyf263 knqyf263 enabled auto-merge November 18, 2024 08:28
@knqyf263 knqyf263 added this pull request to the merge queue Dec 9, 2024
Merged via the queue into aquasecurity:main with commit 9bd6ed7 Dec 9, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants