Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collect xen #464

Merged
merged 34 commits into from
Feb 8, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
c585f02
Merge pull request #2 from nexB/main
Pushpit07 Apr 28, 2021
a2e4487
Merge branch 'nexB:main' into main
Pushpit07 May 22, 2021
46aacf1
Added code to import Xen
Pushpit07 May 27, 2021
c5118c9
Merge branch 'main' into collect_xen
pombredanne Jun 20, 2021
3f776ff
Merge branch 'main' into collect_xen
Pushpit07 Jul 25, 2021
307ead3
Update importer_yielder.py
Pushpit07 Jul 25, 2021
7b828ea
Added xen license info
Pushpit07 Jul 25, 2021
73a884f
helper: split_markdown_front_matter
Hritik14 Apr 18, 2021
a1752bd
black -l 100
Hritik14 Apr 18, 2021
614e0f3
Use split_markdown_front_matter helper in istio
Hritik14 Apr 25, 2021
d872763
Use a more obvious version and reorder imports
Hritik14 Apr 26, 2021
f4ca861
Update according to first review
Hritik14 May 9, 2021
617700a
Comment regarding raw docstring and sort imports
Hritik14 Jun 20, 2021
0f6eece
Remove added_advisories for updated_advisories
Hritik14 Jun 24, 2021
b891459
Speed up test_upstream
Hritik14 Jun 25, 2021
1f2cb59
:see_no_evil: Ignore legacy pypi package versions in github importer
sbs2001 Jun 26, 2021
1f8ef6b
Add basic implementation for time travel in imports
sbs2001 May 30, 2021
e847303
Use web scraping to obtain github tags
sbs2001 Jun 1, 2021
ba9051d
Time travel for githubtags, maven version api importers
sbs2001 Jun 6, 2021
9842367
Fix tests
sbs2001 Jun 8, 2021
0dccf22
Use class for returning versions partitioned by cutoff time
sbs2001 Jun 13, 2021
f7dc3e6
Make review changes
sbs2001 Jul 3, 2021
72a84bc
Refactor package_manangers
Hritik14 Jul 3, 2021
bece29b
Move load_api to parent class and refactor imports
Hritik14 Jul 3, 2021
80e3cf0
Remove misleading frozen dataclass
sbs2001 Jul 17, 2021
9d359f2
Fix npm importer date manipulation
sbs2001 Jul 17, 2021
047398e
Fix recursion in GitHubTagsAPI
Hritik14 Jul 18, 2021
9053ba9
Update importers to handle VersionResponse
Hritik14 Jul 18, 2021
1f3fdef
Update importer_yielder.py
Pushpit07 Jul 25, 2021
5774649
Added xen license info
Pushpit07 Jul 25, 2021
62c2b74
Merge branch 'collect_xen' of https://github.com/Pushpit07/vulnerable…
Pushpit07 Jul 27, 2021
bd18abe
Merge branch 'main' into collect_xen
Hritik14 Feb 7, 2022
8298b4c
Merge branch 'main' of github.com:nexb/vulnerablecode into collect_xen
Hritik14 Feb 8, 2022
6ef62fa
Rename DataSource to Improver, blackify, ignore tests
Hritik14 Feb 8, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pytest.ini
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,6 @@ addopts =
--ignore=vulnerabilities/importers/ubuntu_usn.py
--ignore=vulnerabilities/importers/mozilla.py
--ignore=vulnerabilities/importers/mattermost.py
--ignore=vulnerabilities/importers/xen.py
--ignore=vulnerabilities/management/commands/create_cpe_to_purl_map.py
--ignore=vulnerabilities/lib_oval.py
14 changes: 14 additions & 0 deletions vulnerabilities/fixtures/openssl.json
Original file line number Diff line number Diff line change
Expand Up @@ -40764,5 +40764,19 @@
]
}
}
},
{
"model": "vulnerabilities.importer",
"pk": 17,
"fields": {
"name": "xen",
"license": "gplv2",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know what the license would be?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GPLv2

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Pushpit07

re: GPLv2

if this is it, then state it. Do you have a link to the license info?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"last_run": null,
"data_source": "XenDataSource",
"data_source_cfg": {
"etags": {},
"db_url": "https://xenbits.xen.org/xsa/xsa.json"
}
}
}
]
88 changes: 88 additions & 0 deletions vulnerabilities/importers/xen.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Copyright (c) nexB Inc. and others. All rights reserved.
# http://nexb.com and https://github.com/nexB/vulnerablecode/
# The VulnerableCode software is licensed under the Apache License version 2.0.
# Data generated with VulnerableCode require an acknowledgment.
#
# You may not use this software except in compliance with the License.
# You may obtain a copy of the License at: http://apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software distributed
# under the License is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES OR
# CONDITIONS OF ANY KIND, either express or implied. See the License for the
# specific language governing permissions and limitations under the License.
#
# When you publish or redistribute any data created with VulnerableCode or any VulnerableCode
# derivative work, you must accompany this data with the following acknowledgment:
#
# Generated with VulnerableCode and provided on an 'AS IS' BASIS, WITHOUT WARRANTIES
# OR CONDITIONS OF ANY KIND, either express or implied. No content created from
# VulnerableCode should be considered or used as legal advice. Consult an Attorney
# for any legal advice.
# VulnerableCode is a free software code scanning tool from nexB Inc. and others.
# Visit https://github.com/nexB/vulnerablecode/ for support and download.

import bz2
import dataclasses
import json

import requests
from packageurl import PackageURL

from vulnerabilities.importer import Importer
from vulnerabilities.importer import Advisory
from vulnerabilities.importer import Reference
from vulnerabilities.helpers import create_etag
from vulnerabilities.helpers import is_cve


class XenImporter(Importer):
CONFIG_CLASS = XenDBConfiguration

def updated_advisories(self):
advisories = []
if create_etag(data_src=self, url=self.config.db_url, etag_key="etag"):
advisories.extend(self.to_advisories(fetch(self.config.db_url)))

return self.batch_advisories(advisories)

def create_etag(self, url):
etag = requests.head(url).headers.get("etag")
if not etag:
return True

elif url in self.config.etags:
if self.config.etags[url] == etag:
return False

self.config.etags[url] = etag
return True

@staticmethod
def to_advisories(xen_db):
advisories = []
for xsa in xen_db[0]["xsas"]:
reference = get_xen_references(xsa["xsa"])
title = xsa.get("title", [""])
for cve in xsa.get("cve", [""]):
if not is_cve(cve):
cve = ""

advisories.append(
Advisory(
vulnerability_id=cve,
summary=title,
references=[reference],
)
)
return advisories


def get_xen_references(xsa_id):
return Reference(
reference_id="XSA-" + xsa_id,
url="https://xenbits.xen.org/xsa/advisory-{}.html".format(xsa_id),
)


def fetch(url):
response = requests.get(url).content
return json.loads(response)