Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(PackageURL):add package URL for library scan result #1862

Merged
merged 15 commits into from
Mar 7, 2024

Conversation

TsubasaKanemitsu
Copy link
Contributor

@TsubasaKanemitsu TsubasaKanemitsu commented Mar 5, 2024

What did you implement:

We have updated the version of Trivy used in Vuls to 0.49.1.
As a result, the results of filesystem scan and container image scan now include a Package URL.
Therefore, we will also include Package URL in the results of trivy-to-vuls and lockfile scan.

Type of change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Manually.

trivy-to-vuls

Parse file system scan result

$ trivy -q fs -f json poetry.lock --list-all-pkgs | CGO_ENABLED=0 go run contrib/trivy/cmd/main.go parse --stdin

The PackageURL does not exist in the libraries.Libs.

before
"libraries": [
   {
      "Type": "poetry",
      "Libs": [
         {
            "Name": "anyio",
            "Version": "3.7.1",
            "FilePath": ""
         },
         {
            "Name": "appnope",
            "Version": "0.1.3",
            "FilePath": ""
         },
         {
            "Name": "argon2-cffi",
            "Version": "21.3.0",
            "FilePath": ""
         },
         etc...
      ]
   }
]

The PackageURL is added inside the libraries.Libs.

after
"libraries": [
   {
      "Type": "poetry",
      "Libs": [
         {
            "Name": "anyio",
            "Version": "3.7.1",
            "PackageURL": "pkg:pypi/anyio@3.7.1",
            "FilePath": "",
            "Digest": ""
         },
         {
            "Name": "appnope",
            "Version": "0.1.3",
            "PackageURL": "pkg:pypi/appnope@0.1.3",
            "FilePath": "",
            "Digest": ""
         },
         {
            "Name": "argon2-cffi",
            "Version": "21.3.0",
            "PackageURL": "pkg:pypi/argon2-cffi@21.3.0",
            "FilePath": "",
            "Digest": ""
         },
         etc...
      ]
   }
]

Parse container image scan result

$ trivy -q image -f=json python:3.4-alpine --list-all-pkgs | CGO_ENABLED=0 go run contrib/trivy/cmd/main.go parse --stdin

The PackageURL does not exist in the libraries.Libs.

before
 "libraries": [
    {
       "Type": "python-pkg",
       "Libs": [
          {
             "Name": "pip",
             "Version": "19.0.3",
             "FilePath": "usr/local/lib/python3.4/site-packages/pip-19.0.3.dist-info/METADATA"
          },
          {
             "Name": "setuptools",
             "Version": "40.8.0",
             "FilePath": "usr/local/lib/python3.4/site-packages/setuptools-40.8.0.dist-info/METADATA"
          },
          {
             "Name": "wheel",
             "Version": "0.33.1",
             "FilePath": "usr/local/lib/python3.4/site-packages/wheel-0.33.1.dist-info/METADATA"
          }
       ],
       "path": "Python"
    }
 ],

The PackageURL has been added inside the libraries.Libs.

after
 "libraries": [
    {
       "Type": "python-pkg",
       "Libs": [
          {
             "Name": "pip",
             "Version": "19.0.3",
             "PackageURL": "pkg:pypi/pip@19.0.3",
             "FilePath": "usr/local/lib/python3.4/site-packages/pip-19.0.3.dist-info/METADATA",
             "Digest": ""
          },
          {
             "Name": "setuptools",
             "Version": "40.8.0",
             "PackageURL": "pkg:pypi/setuptools@40.8.0",
             "FilePath": "usr/local/lib/python3.4/site-packages/setuptools-40.8.0.dist-info/METADATA",
             "Digest": ""
          },
          {
             "Name": "wheel",
             "Version": "0.33.1",
             "PackageURL": "pkg:pypi/wheel@0.33.1",
             "FilePath": "usr/local/lib/python3.4/site-packages/wheel-0.33.1.dist-info/METADATA",
             "Digest": ""
          }
       ],
       "path": "Python"
    }
 ],

Lockfile Scan

$ docker run --rm -it \
    -v ~/.ssh:/root/.ssh:ro \
    -v $PWD:/vuls \
    -v $PWD/vuls-log:/var/log/vuls \
    -v /etc/localtime:/etc/localtime:ro \
    -e "TZ=Asia/Tokyo" \
    vuls:latest scan \
    -config=./config.toml

[Mar  5 16:12:45]  INFO [localhost] vuls--build-20240305_160720_584a441
[Mar  5 16:12:45]  INFO [localhost] Start scanning
[Mar  5 16:12:45]  INFO [localhost] config: ./config.toml
[Mar  5 16:12:45]  INFO [localhost] Validating config...
[Mar  5 16:12:45]  INFO [localhost] Detecting Server/Container OS... 
[Mar  5 16:12:45]  INFO [localhost] Detecting OS of servers... 
[Mar  5 16:12:45]  INFO [localhost] (1/1) Detected: localhost: alpine 3.16.9
[Mar  5 16:12:45]  INFO [localhost] Detecting OS of containers... 
[Mar  5 16:12:45]  INFO [localhost] Checking Scan Modes... 
[Mar  5 16:12:45]  INFO [localhost] Detecting Platforms... 
[Mar  5 16:12:45]  INFO [localhost] (1/1) localhost is running on other
[Mar  5 16:12:45]  INFO [localhost] Scanning OS pkg in fast mode
[Mar  5 16:12:47]  INFO [localhost] Scanning listen port...
[Mar  5 16:12:47]  INFO [localhost] Using Port Scanner: Vuls built-in Scanner
[Mar  5 16:12:47]  INFO [localhost] Scanning Language-specific Packages...
[Mar  5 16:12:47]  INFO [localhost] It's recommended to specify FindLockDirs in config.toml. If FindLockDirs is not specified, all directories under / will be searched, which may increase CPU load
[Mar  5 16:12:47]  INFO [localhost] Finding files under /
[Mar  5 16:12:47]  WARN [localhost] Some warnings occurred during scanning on localhost. Please fix the warnings to get a useful information. Execute configtest subcommand before scanning to know the cause of the warnings. warnings: [Standard OS support will be end in 3 months. EOL date: 2024-05-23]


Scan Summary
================
localhost       alpine3.16.9    34 installed, 0 updatable       341 libs

Warning: [Standard OS support will be end in 3 months. EOL date: 2024-05-23]
config.toml
[servers]

[servers.localhost]
host = "localhost"
port = "local"
findLock = true

The PackageURL does not exist in the libraries.Libs.

before
"libraries": [
    {
        "Type": "gomod",
        "Libs": [
            {
                "Name": "cloud.google.com/go",
                "Version": "0.110.10",
                "FilePath": "",
                "Digest": ""
            },
            {
                "Name": "cloud.google.com/go/compute",
                "Version": "1.23.3",
                "FilePath": "",
                "Digest": ""
            },
            {
                "Name": "cloud.google.com/go/compute/metadata",
                "Version": "0.2.3",
                "FilePath": "",
                "Digest": ""
            },
        etc...
        ],
    }
]

The PackageURL has been added inside the libraries.Libs.

after
"libraries": [
      {
          "Type": "gomod",
          "Libs": [
              {
                  "Name": "cloud.google.com/go",
                  "Version": "0.110.10",
                  "PackageURL": "pkg:golang/cloud.google.com/go@0.110.10",
                  "FilePath": "",
                  "Digest": ""
              },
              {
                  "Name": "cloud.google.com/go/compute",
                  "Version": "1.23.3",
                  "PackageURL": "pkg:golang/cloud.google.com/go/compute@1.23.3",
                  "FilePath": "",
                  "Digest": ""
              },
              {
                  "Name": "cloud.google.com/go/compute/metadata",
                  "Version": "0.2.3",
                  "PackageURL": "pkg:golang/cloud.google.com/go/compute/metadata@0.2.3",
                  "FilePath": "",
                  "Digest": ""
              },
              etc...
          ],
     }
]

Checklist:

You don't have to satisfy all of the following.

  • Write tests
  • Write documentation
  • Check that there aren't other open pull requests for the same issue/feature
  • Format your source code by make fmt
  • Pass the test by make test
  • Provide verification config / commands
  • Enable "Allow edits from maintainers" for this PR
  • Update the messages below

Is this ready for review?: YES

Reference

@TsubasaKanemitsu TsubasaKanemitsu changed the title feat(PackageURL):add packagge URL for library scan result [WIP] feat(PackageURL):add packagge URL for library scan result Mar 5, 2024
@TsubasaKanemitsu TsubasaKanemitsu changed the title [WIP] feat(PackageURL):add packagge URL for library scan result [WIP] feat(PackageURL):add package URL for library scan result Mar 6, 2024
@TsubasaKanemitsu TsubasaKanemitsu changed the title [WIP] feat(PackageURL):add package URL for library scan result feat(PackageURL):add package URL for library scan result Mar 6, 2024
@TsubasaKanemitsu TsubasaKanemitsu marked this pull request as ready for review March 6, 2024 11:43
scanner/library.go Outdated Show resolved Hide resolved
scanner/library.go Outdated Show resolved Hide resolved
scanner/library.go Outdated Show resolved Hide resolved
scanner/library.go Outdated Show resolved Hide resolved
contrib/trivy/pkg/converter.go Outdated Show resolved Hide resolved
@MaineK00n MaineK00n self-requested a review March 7, 2024 02:40
Copy link
Collaborator

@shino shino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice feature!

@shino shino merged commit be7b911 into future-architect:master Mar 7, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants