Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dump importer_yielder in favor of IMPORTER_REGISTRY and drop Etags #600

Merged
merged 10 commits into from
Feb 7, 2022

Conversation

Hritik14
Copy link
Collaborator

@Hritik14 Hritik14 commented Jan 26, 2022

The name DataSource and Importer are majorly used interchangeably. This needs to be fixed in a different PR.

Copy link
Collaborator

@pombredanne pombredanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I have a few suggestions and ... Could you add tests?

vulnerabilities/data_source.py Outdated Show resolved Hide resolved
vulnerabilities/data_source.py Outdated Show resolved Hide resolved
vulnerabilities/import_runner.py Outdated Show resolved Hide resolved
vulnerabilities/import_runner.py Show resolved Hide resolved
vulnerabilities/import_runner.py Outdated Show resolved Hide resolved
vulnerabilities/management/commands/import.py Outdated Show resolved Hide resolved
vulnerabilities/management/commands/import.py Show resolved Hide resolved
vulnerabilities/management/commands/import.py Outdated Show resolved Hide resolved
vulnerabilities/management/commands/improve.py Outdated Show resolved Hide resolved
vulnerabilities/models.py Show resolved Hide resolved
@Hritik14 Hritik14 added this to the v30.0 milestone Feb 2, 2022
@Hritik14 Hritik14 force-pushed the migration/importer_yielder branch 5 times, most recently from 89cdfee to 058b9a8 Compare February 5, 2022 21:44
IMPORTER_REGISTRY is neater and does not do any magical string -> object
conversion. The registry looks more in sync with improvers.
Fixes: aboutcode-org#501

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Etags are meant for transient usage in browsers and are not meant for
any long term usage.
Fixes: aboutcode-org#321

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
The name DataSource and Importer were majorly used interchangeably
although they represent the same concept.
Also, the importers now require a mandatory ``spdx_license_expression``
to run

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Same for improvers
IMPROVER_REGISTRY -> IMPROVERS_REGISTRY

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Tests for:
	* manage.py import command
	* manage.py improve command
	* import_runner

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
We can use @classmethod and @Property together in python 3.9. Not so in
3.8

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Copy link
Collaborator

@pombredanne pombredanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks... see a few final nits for your consideration.

vulnerabilities/import_runner.py Outdated Show resolved Hide resolved
vulnerabilities/importer.py Outdated Show resolved Hide resolved
vulnerabilities/importer.py Outdated Show resolved Hide resolved
vulnerabilities/importer.py Outdated Show resolved Hide resolved

# TODO: Needs rewrite
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be an issue to track may work better?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be a part of #597

vulnerabilities/improver.py Outdated Show resolved Hide resolved


def test_ImportRunner_existing_package_and_new_vulnerability(db):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this test be kept somehow? to valid that we never will allow twice the same set of imported data?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assert vuln_refs[0].url == "https://example.com/with/more/info/CVE-2020-13371337"


def test_ImportRunner_new_package_version_affected_by_existing_vulnerability(db):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO this may be need to be kept

assert impacted_package.vulnerability.vulnerability_id == "CVE-2020-13371337"


# def test_ImportRunner_fixed_package_version_is_added(db):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this test have been of value? and need to be kept?

# assert resolved_package.vulnerability.vulnerability_id == "CVE-2020-13371337"


def test_ImportRunner_updated_vulnerability(db):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO this may be need to be kept, moved to improvers?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these will be moved to improvers. Importers cannot test relationships. It will be a part of a new PR. Tracked in: #612

Signed-off-by: Hritik Vijay <hritikxx8@gmail.com>
Co-authored-by: Philippe Ombredanne <pombredanne@gmail.com>
Copy link
Collaborator

@pombredanne pombredanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍

@pombredanne pombredanne merged commit 4ebaa48 into aboutcode-org:main Feb 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Consistent naming of Importer instead of DataSource refactor importer_yielder.py Disable Etag
2 participants