Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: improvements and maintenance #1510

Merged
merged 43 commits into from
Sep 21, 2021
Merged
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
196f39b
ci: update docker image
drptbl Sep 15, 2021
00d9b28
ci: cache improvements
drptbl Sep 15, 2021
e62e3e0
ci: bump ubuntu img
drptbl Sep 15, 2021
c6ded6e
deps: remove codechecks
drptbl Sep 15, 2021
780f6de
chore: rename job-header to job-header-node
drptbl Sep 15, 2021
3a9474e
chore: add job-header-sec-tools
drptbl Sep 15, 2021
53bbe94
ci: use bundled codecov uploader
drptbl Sep 15, 2021
a43b2e9
ci: use bundled codechecks
drptbl Sep 15, 2021
6f83273
ci: add require-audit
drptbl Sep 15, 2021
b7bf92e
ci: add job-audit
drptbl Sep 15, 2021
e40a5d6
ci: add job-audit to the workflow
drptbl Sep 15, 2021
c1f1315
ci: attach workspace in job-prepare
drptbl Sep 15, 2021
fde9510
ci: added audit job
drptbl Sep 15, 2021
89330d7
ci: let codecov search for reports
drptbl Sep 15, 2021
416f0d0
deps: lock all dependencies
drptbl Sep 15, 2021
4ea66b0
Revert "deps: lock all dependencies"
drptbl Sep 15, 2021
14a9d3e
deps: regen package-lock.json
drptbl Sep 15, 2021
5100bb9
fix: codechecks
drptbl Sep 16, 2021
8212faa
ci: codechecks@npm6
drptbl Sep 16, 2021
9367681
Revert "ci: codechecks@npm6"
drptbl Sep 16, 2021
18efb55
Revert "fix: codechecks"
drptbl Sep 16, 2021
1e58094
fix: git-related issue with codechecks
drptbl Sep 16, 2021
12c78b7
deps: use codechecks from inside repository
drptbl Sep 16, 2021
28994aa
ci: fix codechecks only if branch is not master
drptbl Sep 16, 2021
fe1becc
chore: typo fix
drptbl Sep 16, 2021
ab5bcfa
Merge branch 'develop' into update-docker-image
drptbl Sep 16, 2021
453a213
chore: CODECOV_TOKEN not required for public repos
drptbl Sep 16, 2021
9b3783b
chore: use codecov token
drptbl Sep 16, 2021
240386c
test: codecov bash script
drptbl Sep 16, 2021
6d7fb5a
test: use codecov bash script
drptbl Sep 16, 2021
9d6004d
ci: use codecov uploader
drptbl Sep 16, 2021
e46eca7
chore: print codecov logs
drptbl Sep 16, 2021
dc1c7c3
chore: don't fail the job if codecov fails
drptbl Sep 17, 2021
685256f
chore: always pin dependencies versions
drptbl Sep 19, 2021
10d8d2d
ci: post codecov even if CI fails
drptbl Sep 20, 2021
fa15efd
Merge branch 'develop' into ci-improvements
drptbl Sep 20, 2021
712ea78
chore: update codechecks
drptbl Sep 20, 2021
f5378e9
chore: run codecov in non-verbose way
drptbl Sep 20, 2021
c4a55cd
ci: increase timeout for docker chains to boot up
drptbl Sep 20, 2021
7f39d17
ci: use waitretry for wget docker chains fetch
drptbl Sep 20, 2021
0b6e35a
ci: int tests require more resources
drptbl Sep 20, 2021
a8f0f67
Revert "ci: int tests require more resources"
drptbl Sep 20, 2021
b88518d
ci: remove unnecessary sleep, increase timeout to 5 minutes
drptbl Sep 20, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 50 additions & 18 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,27 @@ commands:
wget --retry-connrefused --waitretry=1 --read-timeout=120 --timeout=120 -t 100 http://localhost:<<parameters.port>>
:
jobs:
job-audit:
working_directory: ~/repo
docker:
- image: synthetixio/docker-sec-tools:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
steps:
- checkout
- run:
name: Audit dependencies
command: |
npm audit --audit-level=critical
- run:
name: Lint lockfile
command: |
lockfile-lint -p package-lock.json --type npm --allowed-hosts npm github.com --allowed-schemes "https:" "git+ssh:"
job-compile:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
Expand All @@ -29,7 +46,7 @@ jobs:
job-fork-tests:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
Expand All @@ -54,7 +71,7 @@ jobs:
job-integration-tests:
working_directory: ~/repo
machine:
image: ubuntu-2004:202104-01
image: ubuntu-2004:202107-02
docker_layer_caching: true
resource_class: large
steps:
Expand Down Expand Up @@ -99,7 +116,7 @@ jobs:
job-lint:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
Expand All @@ -111,7 +128,7 @@ jobs:
job-pack-browser:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
Expand All @@ -125,28 +142,37 @@ jobs:
job-prepare:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
steps:
- checkout
- attach_workspace:
at: .
- restore_cache:
keys:
- v4-dependencies-{{ checksum "package-lock.json" }}
- run: npm install
- run:
name: Set custom npm cache directory
command: npm config set cache .npm-cache --global
- run:
name: Install dependencies
command: npm install --prefer-offline --no-audit
- save_cache:
key: v4-dependencies-{{ checksum "package-lock.json" }}
paths:
- node_modules
- .npm-cache
- persist_to_workspace:
root: .
paths:
- node_modules
- .npm-cache
job-simulate-release:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
Expand All @@ -169,7 +195,7 @@ jobs:
job-static-analysis:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
Expand All @@ -186,7 +212,7 @@ jobs:
job-test-deploy-script:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
Expand All @@ -212,7 +238,7 @@ jobs:
job-unit-tests-coverage-report:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-sec-tools:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
Expand All @@ -224,11 +250,11 @@ jobs:
name: Upload coverage
command: |
cp -R /tmp/coverage/coverage-*.json .
bash <(curl -s https://codecov.io/bash)
codecov -t $CODECOV_TOKEN
job-unit-tests-coverage:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
Expand Down Expand Up @@ -258,7 +284,7 @@ jobs:
job-unit-tests-gas-report:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
Expand All @@ -270,13 +296,16 @@ jobs:
name: Upload gas reports
command: |
npx hardhat test:merge-gas-reports gasReporterOutput-*.json
if [ "${CIRCLE_BRANCH}" != "master" ]; then
git branch -f master origin/master
fi
npx codechecks codechecks.unit.yml
- store_artifacts:
path: gasReporterOutput.json
job-unit-tests:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
Expand Down Expand Up @@ -308,7 +337,7 @@ jobs:
job-validate-deployments:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
Expand All @@ -327,7 +356,7 @@ jobs:
job-validate-etherscan:
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
Expand All @@ -342,7 +371,10 @@ jobs:
workflows:
workflow-all:
jobs:
- job-prepare
- job-audit
- job-prepare:
requires:
- job-audit
- job-lint:
requires:
- job-prepare
Expand Down
12 changes: 12 additions & 0 deletions .circleci/src/jobs/job-audit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Bootstraps dependencies
{{> job-header-sec-tools.yml}}
steps:
- checkout
- run:
name: Audit dependencies
command: |
npm audit --audit-level=critical
- run:
name: Lint lockfile
command: |
lockfile-lint -p package-lock.json --type npm --allowed-hosts npm github.com --allowed-schemes "https:" "git+ssh:"
2 changes: 1 addition & 1 deletion .circleci/src/jobs/job-compile.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Compiles all contracts and fails with oversized contracts
{{> job-header.yml}}
{{> job-header-node.yml}}
steps:
- checkout
- attach_workspace:
Expand Down
2 changes: 1 addition & 1 deletion .circleci/src/jobs/job-fork-tests.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Starts a fork of mainnet, deploys the latest release, and runs L1 integration tests
{{> job-header.yml}}
{{> job-header-node.yml}}
steps:
- checkout
- attach_workspace:
Expand Down
2 changes: 1 addition & 1 deletion .circleci/src/jobs/job-lint.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Runs all linters
{{> job-header.yml}}
{{> job-header-node.yml}}
steps:
- checkout
- attach_workspace:
Expand Down
2 changes: 1 addition & 1 deletion .circleci/src/jobs/job-pack-browser.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Packs js code for browser usage
{{> job-header.yml}}
{{> job-header-node.yml}}
steps:
- checkout
- attach_workspace:
Expand Down
13 changes: 11 additions & 2 deletions .circleci/src/jobs/job-prepare.yml
Original file line number Diff line number Diff line change
@@ -1,18 +1,27 @@
# Bootstraps dependencies
{{> job-header.yml}}
{{> job-header-node.yml}}
# set custom delimiter to avoid checksum parsing
{{=<% %>=}}
steps:
- checkout
- attach_workspace:
at: .
- restore_cache:
keys:
- v4-dependencies-{{ checksum "package-lock.json" }}
- run: npm install
- run:
name: Set custom npm cache directory
command: npm config set cache .npm-cache --global
- run:
name: Install dependencies
command: npm install --prefer-offline --no-audit
- save_cache:
key: v4-dependencies-{{ checksum "package-lock.json" }}
paths:
- node_modules
- .npm-cache
- persist_to_workspace:
root: .
paths:
- node_modules
- .npm-cache
2 changes: 1 addition & 1 deletion .circleci/src/jobs/job-simulate-release.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Starts a fork of mainnet, deploys the latest release, and runs L1 integration tests
{{> job-header.yml}}
{{> job-header-node.yml}}
steps:
- checkout
- attach_workspace:
Expand Down
2 changes: 1 addition & 1 deletion .circleci/src/jobs/job-static-analysis.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Runs all static analysis checks
{{> job-header.yml}}
{{> job-header-node.yml}}
steps:
- checkout
- attach_workspace:
Expand Down
2 changes: 1 addition & 1 deletion .circleci/src/jobs/job-test-deploy-script.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Validates that the deploy command is working as expected
{{> job-header.yml}}
{{> job-header-node.yml}}
resource_class: large
steps:
- checkout
Expand Down
4 changes: 2 additions & 2 deletions .circleci/src/jobs/job-unit-tests-coverage-report.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Measures unit and spec test coverage
{{> job-header.yml}}
{{> job-header-sec-tools.yml}}
steps:
- checkout
- attach_workspace:
Expand All @@ -8,4 +8,4 @@ steps:
name: Upload coverage
command: |
cp -R /tmp/coverage/coverage-*.json .
bash <(curl -s https://codecov.io/bash)
codecov -t $CODECOV_TOKEN
2 changes: 1 addition & 1 deletion .circleci/src/jobs/job-unit-tests-coverage.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Measures unit and spec test coverage
{{> job-header.yml}}
{{> job-header-node.yml}}
resource_class: large
parallelism: 8
steps:
Expand Down
6 changes: 5 additions & 1 deletion .circleci/src/jobs/job-unit-tests-gas-report.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Measures deployment and transaction gas usage in unit tests
{{> job-header.yml}}
{{> job-header-node.yml}}
steps:
- checkout
- attach_workspace:
Expand All @@ -8,6 +8,10 @@ steps:
name: Upload gas reports
command: |
npx hardhat test:merge-gas-reports gasReporterOutput-*.json
# required for codechecks
if [ "${CIRCLE_BRANCH}" != "master" ]; then
git branch -f master origin/master
fi
npx codechecks codechecks.unit.yml
- store_artifacts:
path: gasReporterOutput.json
2 changes: 1 addition & 1 deletion .circleci/src/jobs/job-unit-tests.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Runs all unit and spec tests
{{> job-header.yml}}
{{> job-header-node.yml}}
resource_class: large
parallelism: 8
steps:
Expand Down
2 changes: 1 addition & 1 deletion .circleci/src/jobs/job-validate-deployments.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Validates deployment json data against on-chain data
{{> job-header.yml}}
{{> job-header-node.yml}}
steps:
- checkout
- attach_workspace:
Expand Down
2 changes: 1 addition & 1 deletion .circleci/src/jobs/job-validate-etherscan.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Validates that sources have been verified in etherscan
{{> job-header.yml}}
{{> job-header-node.yml}}
parameters:
network:
type: string
Expand Down
2 changes: 1 addition & 1 deletion .circleci/src/snippets/job-header-machine.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
working_directory: ~/repo
machine:
image: ubuntu-2004:202104-01
image: ubuntu-2004:202107-02
docker_layer_caching: true
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
working_directory: ~/repo
docker:
- image: synthetixio/docker-node:14.16-focal
- image: synthetixio/docker-node:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
6 changes: 6 additions & 0 deletions .circleci/src/snippets/job-header-sec-tools.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
working_directory: ~/repo
docker:
- image: synthetixio/docker-sec-tools:14.17-ubuntu
auth:
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
2 changes: 2 additions & 0 deletions .circleci/src/snippets/require-audit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
requires:
- job-audit
4 changes: 3 additions & 1 deletion .circleci/src/workflows/workflow-all.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@ jobs:
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Basic
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- job-prepare
- job-audit
- job-prepare:
{{> require-audit.yml}}
- job-lint:
{{> require-prepare.yml}}
- job-compile:
Expand Down
1 change: 1 addition & 0 deletions .npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
save-exact=true
2 changes: 2 additions & 0 deletions codecov.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
codecov:
require_ci_to_pass: false
Loading