Skip to content

Commit

Permalink
Add Go dependeny scan using golicense
Browse files Browse the repository at this point in the history
We add a script and a corresponding CI job to analyze the Antrea
binaries and extract their Go dependencies. I could not find any good
tool for source-based dependency analysis
(https://github.com/google/go-licenses seems to have many issues, and
fails for quite a lot of depedencies, without any possibility of manual
override). Analyzing binaries also mean that we can avoid some false
positives for transitive dependencies that end up not actually being
used by Antrea.

The CI job will help us automatically detect licensing issues (e.g. non
business-friendly license, such as GPL/LGPL) with new dependencies
before we merge a patch. With the CI integration, we can also publish a
list of the Go dependencies for the Antrea binaries with each release.

TODO: for releases, upload full dependency list to Antrea S3 account.

Fixes antrea-io#345
  • Loading branch information
antoninbas committed Aug 28, 2020
1 parent b2af9f4 commit 2d8c194
Show file tree
Hide file tree
Showing 6 changed files with 211 additions and 0 deletions.
36 changes: 36 additions & 0 deletions .github/workflows/golicense.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Golicense
on:
pull_request:
branches:
- master
- release-*
paths:
- '**go.mod'
push:
branches:
- master
- release-*
release:
types:
- created

jobs:
golicense:
runs-on: [ubuntu-latest]
steps:
- name: Set up Go 1.13
uses: actions/setup-go@v1
with:
go-version: 1.13
- uses: actions/checkout@v2
- run: mkdir antrea-bins license-reports
- name: Build assets
run: |
export VERSION="$(head VERSION)"
./hack/release/prepare-assets.sh ./antrea-bins
- name: Build Linux binaries
run: BINDIR=./antrea-bins make bin
- name: Run golicense
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: ./ci/golicense/run.sh ./antrea-bins ./license-reports
19 changes: 19 additions & 0 deletions build/images/golicense/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
FROM ubuntu:20.04 as install

WORKDIR /

ADD https://github.com/mitchellh/golicense/releases/download/v0.2.0/golicense_0.2.0_linux_x86_64.tar.gz /golicense.tar.gz

WORKDIR /golicense

RUN tar -xzvf /golicense.tar.gz

FROM gcr.io/distroless/base

LABEL maintainer="Antrea <projectantrea-dev@googlegroups.com>"
LABEL description="A Docker image which includes golicense (https://github.com/mitchellh/golicense)"

# Includes a copy of the MIT license
COPY --from=install /golicense /golicense

ENTRYPOINT ["/golicense/golicense"]
16 changes: 16 additions & 0 deletions build/images/golicense/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# images/golicense

This Docker image is a "distroless" image which includes the
[golicense](https://github.com/mitchellh/golicense) tool.

If you need to build a new version of the image and push it to Dockerhub, you
can run the following:

```bash
cd build/images/golicense
docker build -t antrea/golicense .
docker push antrea/golicense
```

The `docker push` command will fail if you do not have permission to push to the
`antrea` Dockerhub repository.
41 changes: 41 additions & 0 deletions ci/golicense/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# OSS License Scanner for Antrea Binaries

The code in this folder integrates with
[golicense](https://github.com/mitchellh/golicense), a command-line tool that
scans compiled Go binaries and outputs all their dependencies with their
respective licenses.

The [run.sh](run.sh) script is meant to be run as a CI job, but can also be run
locally, provided you provide a valid `GITHUB_TOKEN` that will be used by
golicense to make Github API calls:

```bash
GITHUB_TOKEN=<your token here> ./ci/golicense/run.sh <path to Antrea binaries directory> <output directory for generated reports>
```

## Supported OSS Licenses

For a list of the OSS licenses accepted or rejected for Antrea dependencies,
please see [conf.json](conf.json). These lists are not comprehensive and do not
include all possible OSS licenses - however, they do include the most popular
ones. If a patch introduces a new dependency, and the license for that
dependency is listed in "deny", the patch will not be merged. If the license is
neither listed in "deny" nor explicitly listed in "allow", the patch cannot be
merged until project maintainers decide whether the license is accaptable for
the project. [Permissive
licenses](https://en.wikipedia.org/wiki/Permissive_software_license)
which are business-friendly are usually accepted, while [copyleft
licenses](https://en.wikipedia.org/wiki/Copyleft) are not. This is to ensure
that Antrea can easily be used in commercial derivative works.

## Golicense: Binary-based Analysis vs Source-based Analysis

We chose golicense, which uses binary-based dependency analysis, for two
reasons:
* It works very well, and there is no source-based analysis alternative which
is as popular and as easy to use. If we find one, we would consider running
it as well.
* Binary-based analysis avoids "false positives" by ensuring that all the
reported dependencies are actually used in the distributed binary assets. It
is possible for a dependency to be included in the go.mod file, but only used
in tests or code examples.
50 changes: 50 additions & 0 deletions ci/golicense/conf.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
{
"allow": [
"MIT",
"Apache-2.0",
"BSD-2-Clause",
"BSD-3-Clause",
"MPL-2.0",
"ISC",
"EPL-1.0"
],
"deny": [
"GPL-1.0-only",
"GPL-1.0-or-later",
"GPL-1.0",
"GPL-1.0+",
"GPL-2.0-only",
"GPL-2.0-or-later",
"GPL-2.0",
"GPL-2.0+",
"GPL-3.0-only",
"GPL-3.0-or-later",
"GPL-3.0",
"GPL-3.0+",
"LGPL-2.0-only",
"LGPL-2.0-or-later",
"LGPL-2.0",
"LGPL-2.0+",
"LGPL-2.1-only",
"LGPL-2.1-or-later",
"LGPL-2.1",
"LGPL-2.1+",
"LGPL-3.0-only",
"LGPL-3.0-or-later",
"LGPL-3.0",
"LGPL-3.0+",
"MPL-1.0",
"MPL-1.1",
"AGPL-3.0-only",
"AGPL-3.0-or-later",
"AGPL-3.0",
"AGPL-3.0+",
"FTL"
],
"override": {
"github.com/gogo/protobuf": "BSD-3-Clause",
"sigs.k8s.io/yaml": "MIT",
"../../": "Apache-2.0",
"github.com/docker/spdystream": "Apache-2.0"
}
}
49 changes: 49 additions & 0 deletions ci/golicense/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
#!/usr/bin/env bash

set -eo pipefail

if [ "$#" -ne 2 ]; then
echo "Invalid number of parameters. Usage: $0 <PATH TO ANTREA BINARIES DIRECTORY> <OUT PATH FOR REPORTS>"
exit 1
fi

if [ -z "$GITHUB_TOKEN" ]; then
echo "GITHUB_TOKEN environment variable must be set to avoid aggressive API rate limiting"
exit 1
fi

THIS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
BINARIES_DIR="$( cd "$1" >/dev/null 2>&1 && pwd )"
REPORTS_DIR="$( cd "$2" >/dev/null 2>&1 && pwd )"

failed_binaries=""

for f in "$1"/*; do
[ -e "$f" ] || continue
if [[ $f =~ antrea-agent || $f =~ antrea-controller || $f =~ antrea-cni || $f =~ antctl || $f =~ antrea-octant-plugin ]]; then
if [[ $f =~ exe ]]; then
# skip Windows binaries for now
# See https://github.com/mitchellh/golicense/issues/4
continue
fi
base=$(basename $f)
echo "Processing $base"
echo "****************"
docker run --rm -e GITHUB_TOKEN -v $THIS_DIR:/conf -v "$BINARIES_DIR":/bins antrea/golicense /conf/conf.json /bins/$base | tee "$REPORTS_DIR/$base.deps.txt" || failed_binaries="$failed_binaries $base"
echo "****************"
fi
done

echo "Merging all files as $REPORTS_DIR/ALL.deps.txt"
echo "****************"
# The 'grep -v' is to remove the dependency of the Antrea Octant plugin to Antrea
cat "$REPORTS_DIR"/*.deps.txt | grep -v "\.\./\.\." | uniq | tee "$REPORTS_DIR/ALL.deps.txt"
echo "****************"

if [ -z "$failed_binaries" ]; then
echo "#### SUCCESS ####"
else
echo "#### FAILURE ####"
echo "Scan failed for the following binaries: $failed_binaries"
echo "Check $REPORTS_DIR/ALL.deps.txt for more info"
fi

0 comments on commit 2d8c194

Please sign in to comment.