Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor GitlabDataSource to work with browser extension #1524

Merged

Conversation

michaelehab
Copy link
Collaborator

Fix #1523

This PR streamlines the GitLab datasource by eliminating the need for retrieving the repo subtree as a tar.gz file. Instead, it directly fetches YAML files from the repository. This change enhances compatibility, especially in Chrome extensions, and simplifies the codebase.

Changes Made:

  • Removed tar.gz retrieval logic.
  • Implemented direct YAML file fetching.

Testing:

  • Verified functionality in various scenarios, including cli and Chrome extension usage.

@michaelehab michaelehab force-pushed the vulntotal-change-gitlab-datasource branch 2 times, most recently from a3583f4 to 569e2f2 Compare July 24, 2024 15:42
@keshav-space keshav-space added the VulnTotal Tool for cross-validating vulnerability label Jul 29, 2024
Copy link
Member

@keshav-space keshav-space left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @michaelehab!
See some nits below and add DCO to your commits.

vulntotal/datasources/gitlab.py Outdated Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why remove this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the test code, I read the contents of temp_vulntotal_gitlab_datasource folder, and this file is not included in the parsed_advisory-expected so the test was failing when we had it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vulntotal/datasources/gitlab.py Outdated Show resolved Hide resolved
vulntotal/datasources/gitlab.py Show resolved Hide resolved
@michaelehab michaelehab force-pushed the vulntotal-change-gitlab-datasource branch from 3761d09 to f5a1163 Compare August 3, 2024 18:20
Copy link
Member

@keshav-space keshav-space left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaelehab You need to properly rebase or merge with the main branch without modifying the original commits. Let me know if you need help with this.

-Update Gitlab Datasource tests

-Reorder test_parse_interesting_advisories test files

-Format changed files with black

-Merge fetch yml logic and update method doc

Signed-off-by: Michael Ehab Mikhail <michael.ehab@hotmail.com>
@michaelehab michaelehab force-pushed the vulntotal-change-gitlab-datasource branch from e2298be to 8bc0445 Compare August 8, 2024 17:39
@michaelehab
Copy link
Collaborator Author

@keshav-space I squashed the changes and properly merged them with the main branch. Thank you!

Signed-off-by: Michael Ehab Mikhail <michael.ehab@hotmail.com>
Signed-off-by: Michael Ehab Mikhail <michael.ehab@hotmail.com>
Copy link
Member

@keshav-space keshav-space left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @michaelehab, Looking good!

@keshav-space keshav-space merged commit d62f377 into aboutcode-org:main Aug 19, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
VulnTotal Tool for cross-validating vulnerability
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Simplify GitLab Datasource by Directly Fetching YAML Files
2 participants