Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Print package list when extra packages found #1791

Merged
merged 1 commit into from
May 5, 2023

Conversation

willmurphyscode
Copy link
Contributor

@willmurphyscode willmurphyscode commented May 5, 2023

The tests in test/cli/packages_cmd_test.go are hard to debug when different packages are found in different environments. For example, CI runs and M1 macs have been observed to have different package counts. Therefore, if the test is about to fail, log a sorted list of the packages that were found, so that it is easy to compare failures of these tests.

Example of new output (extra logs only happen on failure):

=== RUN   TestPackagesCmdFlags/squashed-scope-flag-hidden-packages
    packages_cmd_test.go:251: expected package count of 163, but found 167
    packages_cmd_test.go:251: package 1: PyGObject:3.22.0
    packages_cmd_test.go:251: package 2: acl:2.2.51-15.el7
    packages_cmd_test.go:251: package 3: audit-libs:2.8.5-4.el7
    packages_cmd_test.go:251: package 4: basesystem:10.0-7.el7.centos
    packages_cmd_test.go:251: package 5: bash:4.2.46-34.el7
    packages_cmd_test.go:251: package 6: bc:1.06.95-13.el7
    packages_cmd_test.go:251: package 7: bind-license:32:9.11.4-26.P2.el7
    packages_cmd_test.go:251: package 8: binutils:2.27-44.base.el7
    packages_cmd_test.go:251: package 9: bzip2-libs:1.0.6-13.el7
    packages_cmd_test.go:251: package 10: ca-certificates:2020.2.41-70.0.el7_8
    ... many more lines

Example of the kind of failure this will help debug: #1788

The tests in test/cli/packages_cmd_test.go are hard to debug when different
packages are found in different environments. For example, CI runs and M1 macs
have been observed to have different package counts. Therefore, if the test is
about to fail, log a sorted list of the packages that were found, so that it is
easy to compare failures of these tests.

Signed-off-by: Will Murphy <will.murphy@anchore.com>
@github-actions
Copy link

github-actions bot commented May 5, 2023

Benchmark Test Results

Benchmark results from the latest changes vs base branch
goos: linux%0Agoarch: amd64%0Apkg: github.com/anchore/syft/test/integration%0Acpu: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz%0A                                                          │ ./.tmp/benchmark-c737f79.txt │%0A                                                          │            sec/op            │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   12.54m ±  1%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    651.9µ ±  1%25%0AImagePackageCatalogers/binary-cataloger-2                                   217.9µ ±  2%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   632.6µ ±  2%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              1.318m ±  3%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         100.7µ ±  2%25%0AImagePackageCatalogers/java-cataloger-2                                     13.74m ± 10%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     100.1µ ±  2%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       425.8µ ±  2%25%0AImagePackageCatalogers/nix-store-cataloger-2                                293.5µ ±  2%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   804.8µ ±  1%25%0AImagePackageCatalogers/portage-cataloger-2                                  399.9µ ±  1%25%0AImagePackageCatalogers/python-package-cataloger-2                           3.294m ±  1%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   562.0µ ±  3%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             934.5µ ±  1%25%0AImagePackageCatalogers/sbom-cataloger-2                                     124.1µ ±  2%25%0Ageomean                                                                     673.1µ%0A%0A                                                          │ ./.tmp/benchmark-c737f79.txt │%0A                                                          │             B/op             │%0AImagePackageCatalogers/alpmdb-cataloger-2                                   5.116Mi ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                    146.7Ki ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                   31.97Ki ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                   170.4Ki ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                              410.6Ki ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                         10.06Ki ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                     2.783Mi ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                     8.750Ki ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                       98.59Ki ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                49.29Ki ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                   179.7Ki ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                  86.40Ki ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                           978.0Ki ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                   178.2Ki ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                             139.7Ki ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                     14.20Ki ± 0%25%0Ageomean                                                                     134.3Ki%0A%0A                                                          │ ./.tmp/benchmark-c737f79.txt │%0A                                                          │          allocs/op           │%0AImagePackageCatalogers/alpmdb-cataloger-2                                    87.63k ± 0%25%0AImagePackageCatalogers/apkdb-cataloger-2                                     3.683k ± 0%25%0AImagePackageCatalogers/binary-cataloger-2                                     896.0 ± 0%25%0AImagePackageCatalogers/dpkgdb-cataloger-2                                    2.997k ± 0%25%0AImagePackageCatalogers/dotnet-deps-cataloger-2                               6.326k ± 0%25%0AImagePackageCatalogers/go-module-binary-cataloger-2                           281.0 ± 0%25%0AImagePackageCatalogers/java-cataloger-2                                      39.47k ± 0%25%0AImagePackageCatalogers/graalvm-native-image-cataloger-2                       228.0 ± 0%25%0AImagePackageCatalogers/javascript-package-cataloger-2                        1.321k ± 0%25%0AImagePackageCatalogers/nix-store-cataloger-2                                  890.0 ± 0%25%0AImagePackageCatalogers/php-composer-installed-cataloger-2                    3.796k ± 0%25%0AImagePackageCatalogers/portage-cataloger-2                                   1.668k ± 0%25%0AImagePackageCatalogers/python-package-cataloger-2                            15.95k ± 0%25%0AImagePackageCatalogers/rpm-db-cataloger-2                                    3.879k ± 0%25%0AImagePackageCatalogers/ruby-gemspec-cataloger-2                              2.279k ± 0%25%0AImagePackageCatalogers/sbom-cataloger-2                                       394.0 ± 0%25%0Ageomean                                                                      2.645k

@willmurphyscode willmurphyscode merged commit 630c18e into main May 5, 2023
@willmurphyscode willmurphyscode deleted the log-packages-on-failure branch May 5, 2023 19:57
spiffcs pushed a commit that referenced this pull request May 10, 2023
The tests in test/cli/packages_cmd_test.go are hard to debug when different
packages are found in different environments. For example, CI runs and M1 macs
have been observed to have different package counts. Therefore, if the test is
about to fail, log a sorted list of the packages that were found, so that it is
easy to compare failures of these tests.

Signed-off-by: Will Murphy <will.murphy@anchore.com>
spiffcs added a commit that referenced this pull request May 10, 2023
* main:
  fix: Reduce log spam on unknown relationship type (#1797)
  chore(deps): update bootstrap tools to latest versions (#1807)
  chore(deps): bump golang.org/x/net from 0.9.0 to 0.10.0 (#1802)
  chore(deps): bump github.com/docker/docker (#1795)
  chore(deps): bump github.com/google/go-containerregistry (#1796)
  chore(deps): update bootstrap tools to latest versions (#1792)
  Print package list when extra packages found (#1791)
  chore(deps): update bootstrap tools to latest versions (#1786)
  chore(deps): bump golang.org/x/term from 0.7.0 to 0.8.0 (#1787)
  Update the CPE generation for spring-security-core (#1789)
  chore: do not HTML escape PackageURLs (#1782)
  chore: do not include kernel module cataloger by default (#1784)

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
spiffcs added a commit that referenced this pull request May 18, 2023
* main: (32 commits)
  chore(deps): bump github.com/google/go-containerregistry (#1823)
  chore(deps): bump github.com/sirupsen/logrus from 1.9.0 to 1.9.1 (#1822)
  chore(deps): bump github.com/docker/docker (#1824)
  fix: update field plurality of 8.0.0 schema before release (#1820)
  fix: update cataloger to check for expressions before split (#1819)
  feat: update syft license concept to complex struct (#1743)
  fix: cyclonedx depends-on relationship inverted (#1816)
  fix: retain sbom cataloger relationships (#1509)
  feat: warn if parsing newer SBOM (#1810)
  feat: Add R cataloger (#1790)
  update cosign to v2 release (different go module) (#1805)
  fix: Reduce log spam on unknown relationship type (#1797)
  chore(deps): update bootstrap tools to latest versions (#1807)
  chore(deps): bump golang.org/x/net from 0.9.0 to 0.10.0 (#1802)
  chore(deps): bump github.com/docker/docker (#1795)
  chore(deps): bump github.com/google/go-containerregistry (#1796)
  chore(deps): update bootstrap tools to latest versions (#1792)
  Print package list when extra packages found (#1791)
  chore(deps): update bootstrap tools to latest versions (#1786)
  chore(deps): bump golang.org/x/term from 0.7.0 to 0.8.0 (#1787)
  ...

Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
GijsCalis pushed a commit to GijsCalis/syft that referenced this pull request Feb 19, 2024
The tests in test/cli/packages_cmd_test.go are hard to debug when different
packages are found in different environments. For example, CI runs and M1 macs
have been observed to have different package counts. Therefore, if the test is
about to fail, log a sorted list of the packages that were found, so that it is
easy to compare failures of these tests.

Signed-off-by: Will Murphy <will.murphy@anchore.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants