forked from antrea-io/antrea
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add Go dependeny scan using golicense
We add a script and a corresponding CI job to analyze the Antrea binaries and extract their Go dependencies. I could not find any good tool for source-based dependency analysis (https://github.com/google/go-licenses seems to have many issues, and fails for quite a lot of depedencies, without any possibility of manual override). Analyzing binaries also mean that we can avoid some false positives for transitive dependencies that end up not actually being used by Antrea. The CI job will help us automatically detect licensing issues (e.g. non business-friendly license, such as GPL/LGPL) with new dependencies before we merge a patch. With the CI integration, we can also publish a list of the Go dependencies for the Antrea binaries with each release. TODO: for releases, upload full dependency list to Antrea S3 account. Fixes antrea-io#345
- Loading branch information
1 parent
2ec50c0
commit aac26c3
Showing
6 changed files
with
211 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
name: Golicense | ||
on: | ||
pull_request: | ||
branches: | ||
- master | ||
- release-* | ||
paths: | ||
- '**go.mod' | ||
push: | ||
branches: | ||
- master | ||
- release-* | ||
release: | ||
types: | ||
- created | ||
|
||
jobs: | ||
golicense: | ||
runs-on: [ubuntu-latest] | ||
steps: | ||
- name: Set up Go 1.13 | ||
uses: actions/setup-go@v1 | ||
with: | ||
go-version: 1.13 | ||
- uses: actions/checkout@v2 | ||
- run: mkdir antrea-bins license-reports | ||
- name: Build assets | ||
run: | | ||
export VERSION="$(head VERSION)" | ||
./hack/release/prepare-assets.sh ./antrea-bins | ||
- name: Build Linux binaries | ||
run: BINDIR=./antrea-bins make bin | ||
- name: Run golicense | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
run: ./ci/golicense/run.sh ./antrea-bins ./license-reports |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
FROM ubuntu:20.04 as install | ||
|
||
WORKDIR / | ||
|
||
ADD https://github.com/mitchellh/golicense/releases/download/v0.2.0/golicense_0.2.0_linux_x86_64.tar.gz /golicense.tar.gz | ||
|
||
WORKDIR /golicense | ||
|
||
RUN tar -xzvf /golicense.tar.gz | ||
|
||
FROM gcr.io/distroless/base | ||
|
||
LABEL maintainer="Antrea <projectantrea-dev@googlegroups.com>" | ||
LABEL description="A Docker image which includes golicense (https://github.com/mitchellh/golicense)" | ||
|
||
# Includes a copy of the MIT license | ||
COPY --from=install /golicense /golicense | ||
|
||
ENTRYPOINT ["/golicense/golicense"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
# images/golicense | ||
|
||
This Docker image is a "distroless" image which includes the | ||
[golicense](https://github.com/mitchellh/golicense) tool. | ||
|
||
If you need to build a new version of the image and push it to Dockerhub, you | ||
can run the following: | ||
|
||
```bash | ||
cd build/images/golicense | ||
docker build -t antrea/golicense . | ||
docker push antrea/golicense | ||
``` | ||
|
||
The `docker push` command will fail if you do not have permission to push to the | ||
`antrea` Dockerhub repository. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
# OSS License Scanner for Antrea Binaries | ||
|
||
The code in this folder integrates with | ||
[golicense](https://github.com/mitchellh/golicense), a command-line tool that | ||
scans compiled Go binaries and outputs all their dependencies with their | ||
respective licenses. | ||
|
||
The [run.sh](run.sh) script is meant to be run as a CI job, but can also be run | ||
locally, provided you provide a valid `GITHUB_TOKEN` that will be used by | ||
golicense to make Github API calls: | ||
|
||
```bash | ||
GITHUB_TOKEN=<your token here> ./ci/golicense/run.sh <path to Antrea binaries directory> <output directory for generated reports> | ||
``` | ||
|
||
## Supported OSS Licenses | ||
|
||
For a list of the OSS licenses accepted or rejected for Antrea dependencies, | ||
please see [conf.json](conf.json). These lists are not comprehensive and do not | ||
include all possible OSS licenses - however, they do include the most popular | ||
ones. If a patch introduces a new dependency, and the license for that | ||
dependency is listed in "deny", the patch will not be merged. If the license is | ||
neither listed in "deny" nor explicitly listed in "allow", the patch cannot be | ||
merged until project maintainers decide whether the license is acceptable for | ||
the project. [Permissive | ||
licenses](https://en.wikipedia.org/wiki/Permissive_software_license) | ||
which are business-friendly are usually accepted, while [copyleft | ||
licenses](https://en.wikipedia.org/wiki/Copyleft) are not. This is to ensure | ||
that Antrea can easily be used in commercial derivative works. | ||
|
||
## Golicense: Binary-based Analysis vs Source-based Analysis | ||
|
||
We chose golicense, which uses binary-based dependency analysis, for two | ||
reasons: | ||
* It works very well, and there is no source-based analysis alternative which | ||
is as popular and as easy to use. If we find one, we would consider running | ||
it as well. | ||
* Binary-based analysis avoids "false positives" by ensuring that all the | ||
reported dependencies are actually used in the distributed binary assets. It | ||
is possible for a dependency to be included in the go.mod file, but only used | ||
in tests or code examples. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
{ | ||
"allow": [ | ||
"MIT", | ||
"Apache-2.0", | ||
"BSD-2-Clause", | ||
"BSD-3-Clause", | ||
"MPL-2.0", | ||
"ISC", | ||
"EPL-1.0" | ||
], | ||
"deny": [ | ||
"GPL-1.0-only", | ||
"GPL-1.0-or-later", | ||
"GPL-1.0", | ||
"GPL-1.0+", | ||
"GPL-2.0-only", | ||
"GPL-2.0-or-later", | ||
"GPL-2.0", | ||
"GPL-2.0+", | ||
"GPL-3.0-only", | ||
"GPL-3.0-or-later", | ||
"GPL-3.0", | ||
"GPL-3.0+", | ||
"LGPL-2.0-only", | ||
"LGPL-2.0-or-later", | ||
"LGPL-2.0", | ||
"LGPL-2.0+", | ||
"LGPL-2.1-only", | ||
"LGPL-2.1-or-later", | ||
"LGPL-2.1", | ||
"LGPL-2.1+", | ||
"LGPL-3.0-only", | ||
"LGPL-3.0-or-later", | ||
"LGPL-3.0", | ||
"LGPL-3.0+", | ||
"MPL-1.0", | ||
"MPL-1.1", | ||
"AGPL-3.0-only", | ||
"AGPL-3.0-or-later", | ||
"AGPL-3.0", | ||
"AGPL-3.0+", | ||
"FTL" | ||
], | ||
"override": { | ||
"github.com/gogo/protobuf": "BSD-3-Clause", | ||
"sigs.k8s.io/yaml": "MIT", | ||
"../../": "Apache-2.0", | ||
"github.com/docker/spdystream": "Apache-2.0" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
#!/usr/bin/env bash | ||
|
||
set -eo pipefail | ||
|
||
if [ "$#" -ne 2 ]; then | ||
echo "Invalid number of parameters. Usage: $0 <PATH TO ANTREA BINARIES DIRECTORY> <OUT PATH FOR REPORTS>" | ||
exit 1 | ||
fi | ||
|
||
if [ -z "$GITHUB_TOKEN" ]; then | ||
echo "GITHUB_TOKEN environment variable must be set to avoid aggressive API rate limiting" | ||
exit 1 | ||
fi | ||
|
||
THIS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )" | ||
BINARIES_DIR="$( cd "$1" >/dev/null 2>&1 && pwd )" | ||
REPORTS_DIR="$( cd "$2" >/dev/null 2>&1 && pwd )" | ||
|
||
failed_binaries="" | ||
|
||
for f in "$1"/*; do | ||
[ -e "$f" ] || continue | ||
if [[ $f =~ antrea-agent || $f =~ antrea-controller || $f =~ antrea-cni || $f =~ antctl || $f =~ antrea-octant-plugin ]]; then | ||
if [[ $f =~ exe ]]; then | ||
# skip Windows binaries for now | ||
# See https://github.com/mitchellh/golicense/issues/4 | ||
continue | ||
fi | ||
base=$(basename $f) | ||
echo "Processing $base" | ||
echo "****************" | ||
docker run --rm -e GITHUB_TOKEN -v $THIS_DIR:/conf -v "$BINARIES_DIR":/bins antrea/golicense /conf/conf.json /bins/$base | tee "$REPORTS_DIR/$base.deps.txt" || failed_binaries="$failed_binaries $base" | ||
echo "****************" | ||
fi | ||
done | ||
|
||
echo "Merging all files as $REPORTS_DIR/ALL.deps.txt" | ||
echo "****************" | ||
# The 'grep -v' is to remove the dependency of the Antrea Octant plugin to Antrea | ||
cat "$REPORTS_DIR"/*.deps.txt | grep -v "\.\./\.\." | uniq | tee "$REPORTS_DIR/ALL.deps.txt" | ||
echo "****************" | ||
|
||
if [ -z "$failed_binaries" ]; then | ||
echo "#### SUCCESS ####" | ||
else | ||
echo "#### FAILURE ####" | ||
echo "Scan failed for the following binaries: $failed_binaries" | ||
echo "Check $REPORTS_DIR/ALL.deps.txt for more info" | ||
fi |