Skip to content

Commit

Permalink
chore(report) : Coding period week 3 and week 4 reports
Browse files Browse the repository at this point in the history
Add Screenshots and coding period week 3 and week 4 reports

Signed-off-by: Rajul Jha <rajuljha49@gmail.com>
  • Loading branch information
rajuljha committed Jun 27, 2024
1 parent f72a153 commit 754e623
Show file tree
Hide file tree
Showing 7 changed files with 112 additions and 0 deletions.
51 changes: 51 additions & 0 deletions docs/2024/ci-scanner/updates/2024-06-20.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
title: Week 3
author: Rajul Jha
---
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
SPDX-FileCopyrightText: 2024 Rajul Jha <rajuljha49gmail.com>
-->

# Week 3
*(June 14, 2024 - June 20, 2024)*

## Meeting 1
*(June 19, 2024)*

## Attendees
* [Rajul Jha](https://github.com/rajuljha)
* [Kaushlendra](https://github.com/Kaushl2208)
* [Shaheem Azmal](https://github.com/shaheemazmalmmd)
* Katharina


## Discussions

* Worked on adding nomos json output. But not able to access the theMatches object's licenseAndMatchPositions object. Stuck on this for nomos task.
* Bumped up spdx_tools library to latest version. Tested with both platforms (GH Actions and Gitlab).
* Discussed an issue in the github workflow, not supporting multiple architectures.
* The GH Actions Runner does not support multi-architecture images.
* We discussed potential solutions for the same including trying to build the image in the GH Action itself or utilizing an emulator like qemu.

## Work Done

* Upgraded the spdx_tools library [(#PR2762)](https://github.com/fossology/fossology/pull/2762) and did performance analysis for both versions.
* With Version `0.0.0a2`:
![Screenshot](/img/ci/Spdx_tools_Version0.8.0a2.png)

* With Version `0.8.2`:
![Screenshot](/img/ci/Spdx_tools_Version_0.8.2.png)

* Worked on providing custom `keyword.conf` file during CI pipeline. It works as follows:
* User creates a custom `keyword.conf` file following [this](https://github.com/fossology/fossology/blob/master/src/copyright/agent/keyword.conf) pattern.
* They set an environment variable called `KEYWORD_CONF_FILE_PATH` in CI providers settings and set its value to the path of the `keyword.conf` file.
* Then, when the pipeline is triggered, say on `push` to `main`, then the keyword scanner reads the custom file and scans for the keywords specified by the user.
* Keep in mind that the current keyword file is overwritten by the script, as discussed with the mentors.

## Planning for next week

* Test the keyword functionality with GH Actions and Gitlab CI and send out a PR for the same.
* Work on providing multi architecture support for GH Actions.
* Study about how to implement differential scans.
61 changes: 61 additions & 0 deletions docs/2024/ci-scanner/updates/2024-06-27.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
title: Week 4
author: Rajul Jha
---
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
SPDX-FileCopyrightText: 2024 Rajul Jha <rajuljha49gmail.com>
-->

# Week 4
*(June 21, 2024 - June 27, 2024)*

## Meeting 1
*(June 26, 2024)*

## Attendees
* [Rajul Jha](https://github.com/rajuljha)
* [Kaushlendra](https://github.com/Kaushl2208)
* [Shaheem Azmal](https://github.com/shaheemazmalmmd)
* [Avinal Kumar](https://github.com/avinal)
* Katharina

## Discussions

* Talked about how the differential scans need to work in detail, like:
* Should they be triggered on `release` only? As it makes sense to scan between two different versions when a new release it being published.
* Or should the user be able to scan between two different versions via any sort of trigger, like on `push` or `commit` as well?
* Talked about the current implementation of custom keyword functionality, and how it can be
extrapolated similarly for different scanners also.
* We talked about fleshing out the design and implementation details first, then work on improving speed.

## Work Done

* Completed custom keyword functionality. [(#PR2769)](https://github.com/fossology/fossology/pull/2769) The details are as follows:
* User can now setup a GH Action or Gitlab CI pipeline and pass environment variables to scan for custom keywords however they want. This gives them immense control over what they wan't to scan.
* Added some validity checks for the `keyword.conf` file as during testing, I found out that the scanner breaks if an invalid file is passed to it.
* Set up a test docker image [(rjknightmare/fo-ci-test:latest)](https://hub.docker.com/layers/rjknightmare/fo-ci-test/latest/images/sha256-ac72768b8c4383a4aca05a50cd53ae427db9645b42619ad82b53e38b3b876b38?context=repo) for testing.
Here are the test results for the same:
* GH Actions:

![Screenshot](/img/ci/GH_Actions_1.png)

* Gitlab CI:

![Screenshot](/img/ci/Gitlab_1.png)

* Set up qemu emulator for the Github Actions environment to provide multiple architecture support, which also reduced the time for action execution significantly.

![Screenshot](/img/ci/GH_Actions_2.png)

* Studied about how we can implement differential (between two different versions) scans and explored these use cases:
* Use Case I: On `release`
* Use Case II: On `push`, `commit` or any other event trigger.


## Planning for next week

* Study further on above use cases and work out a plan for the same.
* Explore the GH Actions and Gitlab API's and use the best endpoints for our use cases.
* Then start working on implementation of differential scans.
Binary file added static/img/ci/GH_Actions_1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/ci/GH_Actions_2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/ci/Gitlab_1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/ci/Spdx_tools_Version0.8.0a2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/ci/Spdx_tools_Version_0.8.2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 754e623

Please sign in to comment.