Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AJ-1094: Initial Project Setup and Hello World CLI #1

Merged
merged 46 commits into from
Jul 20, 2023
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
e877db0
terra-java-project-template
snf2ye Jun 20, 2023
ccd077c
Remove example repo code
snf2ye Jun 21, 2023
7194f92
rename javatemplate to javapfb
snf2ye Jun 20, 2023
67b3e75
Update GHA's
snf2ye Jun 21, 2023
1008712
Hello world CLI
snf2ye Jun 21, 2023
8d530f6
Another round of template deletions
snf2ye Jun 22, 2023
7fd15a5
rename service to lib directory
snf2ye Jun 22, 2023
d0cc905
update github workflows
snf2ye Jun 23, 2023
ffbbc44
Try slightly different jfrog import
snf2ye Jun 23, 2023
8324e8f
Update LICENSE
snf2ye Jul 10, 2023
2fb38b5
Update release version
snf2ye Jul 10, 2023
faf6fa1
Update gradle version
snf2ye Jul 10, 2023
b909cd9
Remove spring; rename lib to library
snf2ye Jul 11, 2023
2e041ac
rework build.gradle
snf2ye Jul 13, 2023
41b8068
Update build/test GHA: Remove python check; add best test and sonar s…
snf2ye Jul 14, 2023
dac7e43
update sonar configuration
snf2ye Jul 14, 2023
834ce97
info
snf2ye Jul 14, 2023
460bece
Update readme
snf2ye Jul 14, 2023
fc87d7f
switch back to sonarqube command
snf2ye Jul 14, 2023
b693257
same exact setup as data catalog
snf2ye Jul 14, 2023
d494492
update project key
snf2ye Jul 14, 2023
86bc394
Update library/build.gradle
TomConner Jul 14, 2023
81b7874
switch to sonar command and upgrade to v4
snf2ye Jul 14, 2023
22927ac
Dependency cleanup
snf2ye Jul 17, 2023
7b4294a
Run sonar both CLI and library
snf2ye Jul 18, 2023
6b41911
Remove spotbugs: Sonar is sufficient to meet to appsec requirement
snf2ye Jul 18, 2023
6fc95f3
Just run sonar in library for now
snf2ye Jul 18, 2023
93ce6cf
ignore publish action; will fix with AJ-1095
snf2ye Jul 18, 2023
7371bb2
Update main method so it's not empty
snf2ye Jul 18, 2023
397ae31
update fatjar task to just be named jar
snf2ye Jul 18, 2023
9dc57a0
Update readme
snf2ye Jul 18, 2023
f519144
stub out some library code
snf2ye Jul 18, 2023
cda1397
rework package naming/structure
snf2ye Jul 18, 2023
c07d89c
update to new setup method of picocli
snf2ye Jul 18, 2023
e66e9ab
set up help and version message for CLI
snf2ye Jul 19, 2023
fb48d59
Add some test coverage for CLI
snf2ye Jul 19, 2023
79ad761
bare minimum description for library
snf2ye Jul 19, 2023
41b05de
update cli readme
snf2ye Jul 19, 2023
4870da5
Try running sonar for both CLI and library
snf2ye Jul 19, 2023
e90b6cc
Remove unneeded gradle conventions file
snf2ye Jul 19, 2023
b18f455
separate step for cli and library sonar scans
snf2ye Jul 19, 2023
82642ac
upgrade gradle to 8.2.1
snf2ye Jul 20, 2023
2017fad
Remove cli test command
snf2ye Jul 20, 2023
f484b35
Set srcclr scope to runtime
snf2ye Jul 20, 2023
0aecde2
Add a little more context to our source clear and sonar cloud runs.
snf2ye Jul 20, 2023
868ab14
Only need test action in common definitions; don't need jfrog def
snf2ye Jul 20, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 142 additions & 0 deletions .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
name: Build and Test

on:
push:
branches: [ main ]
paths-ignore: [ '*.md' ]
pull_request:
branches: [ '**' ]

jobs:
build:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: Install black and link shellcheck into expected location
run: |
snf2ye marked this conversation as resolved.
Show resolved Hide resolved
pip install black --force-reinstall black==22.3.0
sudo ln -s $(which shellcheck) /usr/local/bin/shellcheck
- name: Set up JDK
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'
cache: 'gradle'

- name: Build all projects without running tests
run: ./gradlew --build-cache build -x test

- name: Upload spotbugs results
uses: github/codeql-action/upload-sarif@main
with:
sarif_file: lib/build/reports/spotbugs/main.sarif

jib:
needs: [ build ]
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Set up JDK
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'
cache: 'gradle'

- name: Construct docker image name and tag
id: image-name
run: |
GITHUB_REPO=$(basename ${{ github.repository }})
GIT_SHORT_HASH=$(git rev-parse --short HEAD)
echo "name=${GITHUB_REPO}:${GIT_SHORT_HASH}" >> $GITHUB_OUTPUT

- name: Build image locally with jib
run: |
./gradlew --build-cache :lib:jibDockerBuild \
--image=${{ steps.image-name.outputs.name }} \
-Djib.console=plain

dispatch-trivy:
needs: [ build ]
runs-on: ubuntu-latest

if: github.event_name == 'pull_request'

steps:
- name: Fire off Trivy action
uses: broadinstitute/workflow-dispatch@v1
with:
workflow: Trivy
token: ${{ secrets.BROADBOT_TOKEN }}
ref: ${{ github.event.pull_request.head.ref }}

source-clear:
needs: [ build ]
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Set up JDK
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'
cache: 'gradle'

- name: SourceClear scan
env:
SRCCLR_API_TOKEN: ${{ secrets.SRCCLR_API_TOKEN }}
run: ./gradlew --build-cache srcclr

unit-tests-and-sonarqube:
needs: [ build ]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
# Needed by sonar to get the git history for the branch the PR will be merged into.
with:
fetch-depth: 0
- name: Set up JDK
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'
cache: 'gradle'

notify-slack:
needs: [ build, unit-tests-and-sonarqube, source-clear ]
runs-on: ubuntu-latest

if: failure() && github.ref == 'refs/heads/main'

steps:
- name: Notify slack on failure
uses: broadinstitute/action-slack@v3.8.0
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
with:
channel: '#dsp-analysis-journeys-alerts'
status: failure
author_name: Build on dev
fields: job,message
text: 'Build failed :sadpanda:'
username: 'Java-PFB GitHub Action'

dispatch-tag:
needs: [ build, unit-tests-and-sonarqube, source-clear ]
runs-on: ubuntu-latest

if: success() && github.ref == 'refs/heads/main'

steps:
- name: Fire off tag action
uses: broadinstitute/workflow-dispatch@v1
with:
workflow: Tag
token: ${{ secrets.BROADBOT_TOKEN }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we will want to tag on every push to main - but I'm totally fine leaving this as-is and addressing in the future, once we have a better idea of what we DO want

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok! I'm inclined to tag on every push to main, but agreed - let's leave as-is and we'll address it in the future once we've had a chance to discuss!

48 changes: 48 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Publish and deploy
on: create

env:
SERVICE_NAME: ${{ github.event.repository.name }}
GOOGLE_PROJECT: broad-dsp-gcr-public

jobs:
publish-job:
if: startsWith(github.ref, 'refs/tags/')
permissions:
contents: 'read'
id-token: 'write'
runs-on: ubuntu-latest
outputs:
tag: ${{ steps.tag.outputs.tag }}
steps:
- uses: actions/checkout@v3
- name: Set up JDK
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'
cache: 'gradle'

- name: Parse tag
id: tag
run: echo "tag=$(git describe --tags)" >> $GITHUB_OUTPUT

- name: Publish to Artifactory
run: ./gradlew --build-cache :client:artifactoryPublish
pshapiro4broad marked this conversation as resolved.
Show resolved Hide resolved
env:
ARTIFACTORY_USERNAME: ${{ secrets.ARTIFACTORY_USERNAME }}
ARTIFACTORY_PASSWORD: ${{ secrets.ARTIFACTORY_PASSWORD }}
ARTIFACTORY_REPO_KEY: "libs-release-local"

- name: Notify slack on failure
uses: broadinstitute/action-slack@v3.8.0
if: failure()
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
with:
channel: '#dsp-analysis-journeys-alerts'
status: failure
author_name: Publish to dev
fields: job
text: 'Publish failed :sadpanda:'
username: 'Java-PFB GitHub Action'
22 changes: 22 additions & 0 deletions .github/workflows/tag.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
name: Tag
on: workflow_dispatch

jobs:
tag-job:
runs-on: ubuntu-latest
steps:
- name: Checkout current code
uses: actions/checkout@v3
with:
token: ${{ secrets.BROADBOT_TOKEN }} # this allows the push to succeed later
- name: Bump the tag to a new version
# https://github.com/DataBiosphere/github-actions/tree/master/actions/bumper
uses: databiosphere/github-actions/actions/bumper@bumper-0.0.6
id: tag
env:
GITHUB_TOKEN: ${{ secrets.BROADBOT_TOKEN }}
HOTFIX_BRANCHES: hotfix.*
DEFAULT_BUMP: minor
RELEASE_BRANCHES: main
VERSION_FILE_PATH: settings.gradle
VERSION_LINE_MATCH: "^\\s*gradle.ext.releaseVersion\\s*=\\s*'.*'"
51 changes: 51 additions & 0 deletions .github/workflows/trivy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Trivy
on: workflow_dispatch

jobs:
trivy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- name: Set up JDK
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'
cache: 'gradle'

- name: Build all projects without running tests
run: ./gradlew --build-cache build -x test -x spotlessCheck

- name: Construct docker image name and tag
id: image-name
run: |
echo "name=trivy-local-testing-image" >> $GITHUB_OUTPUT

- name: Build image locally with jib
run: |
./gradlew --build-cache :service:jibDockerBuild \
snf2ye marked this conversation as resolved.
Show resolved Hide resolved
--image=${{ steps.image-name.outputs.name }} \
-Djib.console=plain

- name: Run Trivy vulnerability scanner
uses: broadinstitute/dsp-appsec-trivy-action@v1
with:
image: ${{ steps.image-name.outputs.name }}

notify-slack:
needs: [ trivy ]
runs-on: ubuntu-latest
if: failure()
steps:
- name: Notify slack on failure
uses: broadinstitute/action-slack@v3.8.0
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
with:
channel: '#dsp-analysis-journeys-alerts'
status: failure
author_name: Trivy action
fields: workflow,message
text: 'Trivy scan failure :sadpanda:'
username: 'Java-PFB GitHub Action'
38 changes: 38 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
HELP.md
.gradle
build/
!gradle/wrapper/gradle-wrapper.jar
!**/src/main/**/build/
!**/src/test/**/build/
bootrun.log

### STS ###
.apt_generated
.classpath
.factorypath
.project
.settings
.springBeans
.sts4-cache
bin/
!**/src/main/**/bin/
!**/src/test/**/bin/

# Emacs backup files #
*.*~

### IntelliJ IDEA ###
.idea/
*.iml

### VS Code ###
.vscode/

# Mac directory metadata
.DS_Store

# PyEnv environment files
.env/

# Ignore generated credentials from google-github-actions/auth
gha-creds-*.json
29 changes: 29 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
BSD 3-Clause License

Copyright (c) 2022, Broad Institute
snf2ye marked this conversation as resolved.
Show resolved Hide resolved
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30 changes: 30 additions & 0 deletions buildSrc/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
plugins {
id 'groovy-gradle-plugin'
}

repositories {
mavenCentral()
maven {
url 'https://broadinstitute.jfrog.io/broadinstitute/libs-release-local/'
}
maven {
url 'https://broadinstitute.jfrog.io/broadinstitute/libs-snapshot-local/'
}
gradlePluginPortal()
}

dependencies {
implementation 'com.diffplug.spotless:spotless-plugin-gradle:6.11.0'
implementation 'com.felipefzdz.gradle.shellcheck:shellcheck:1.4.6'
implementation 'com.google.cloud.tools.jib:com.google.cloud.tools.jib.gradle.plugin:3.3.0'
snf2ye marked this conversation as resolved.
Show resolved Hide resolved
implementation 'com.srcclr.gradle:com.srcclr.gradle.gradle.plugin:3.1.12'
implementation 'de.undercouch.download:de.undercouch.download.gradle.plugin:5.2.0'
snf2ye marked this conversation as resolved.
Show resolved Hide resolved
implementation 'com.github.spotbugs.snom:spotbugs-gradle-plugin:5.0.12'
snf2ye marked this conversation as resolved.
Show resolved Hide resolved
implementation 'io.spring.dependency-management:io.spring.dependency-management.gradle.plugin:1.0.14.RELEASE'
implementation 'org.hidetake.swagger.generator:org.hidetake.swagger.generator.gradle.plugin:2.19.2'
implementation 'org.sonarqube:org.sonarqube.gradle.plugin:3.4.0.2513'
implementation 'org.springframework.boot:spring-boot-gradle-plugin:2.7.0'
implementation 'bio.terra:terra-test-runner:0.1.5-SNAPSHOT'
// This is required due to a dependency conflict between jib and srcclr. Removing it will cause jib to fail.
implementation 'org.apache.commons:commons-compress:1.21'
snf2ye marked this conversation as resolved.
Show resolved Hide resolved
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The filename here is incorrect, I think. You want your plugin name to be scoped to a name specific to this project. So the filename (and plugin name, when used) should be something like bio.terra.pfb.java-application-conventions.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I've renamed these convention files

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
plugins {
// Apply the common convention plugin for shared build configuration between library and application projects.
id 'bio.terra.java-common-conventions'

id 'application'
}
Loading
Loading