Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): make Coverage CI run successfully #3521

Merged
merged 143 commits into from
Dec 4, 2024
Merged
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
143 commits
Select commit Hold shift + click to select a range
2f565a2
disable swap space allocation
Alex6323 Oct 22, 2024
c34139c
update config-patch
Alex6323 Oct 22, 2024
a53a435
temporarily disable report upload
Alex6323 Oct 22, 2024
5b1e2eb
upload coverage reports as Github artifacts
Alex6323 Oct 23, 2024
bc099a0
fix path
Alex6323 Oct 23, 2024
69585a8
update Cargo.lock
Alex6323 Oct 23, 2024
6e2797c
temporarily disable slack bot notificaiton
Alex6323 Oct 23, 2024
4fb0695
update config-patch
Alex6323 Oct 24, 2024
eaf571b
clean up artifacts
Alex6323 Oct 24, 2024
7e66a31
also clean in between
Alex6323 Oct 24, 2024
ab58f70
fix
Alex6323 Oct 24, 2024
90c47e5
investigate seg faults during simtest
Alex6323 Oct 24, 2024
704fb0c
run only simtest
Alex6323 Oct 25, 2024
cdc8b9a
disable simtest; update grcov workflow
Alex6323 Oct 28, 2024
195f1a4
cleanup
Alex6323 Oct 28, 2024
9737f6b
clean up
Alex6323 Oct 29, 2024
c9b36b3
nightly why not
Alex6323 Oct 29, 2024
2eb8746
archive html coverage report
Alex6323 Oct 29, 2024
01876b6
test archiving
Alex6323 Oct 30, 2024
8f8317f
Merge branch 'upstream/develop' into dev-tools/reenable-coverage-ci
Alex6323 Oct 30, 2024
41e7094
restructure
Alex6323 Oct 30, 2024
716dd4c
try release build
Alex6323 Oct 30, 2024
17e34de
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Oct 30, 2024
4ea7fe9
undo release build, undo restructure
Alex6323 Oct 30, 2024
344a3b6
try the hippie
Alex6323 Oct 30, 2024
07e005b
fix path; include branch coverage again
Alex6323 Oct 30, 2024
e4a587e
only simtests one-by-one
Alex6323 Oct 30, 2024
41c09c6
only simtests one-by-one 2; increase retries
Alex6323 Oct 30, 2024
7016112
exclude segfaulting crates from coverage
Alex6323 Oct 31, 2024
b91e9cb
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Oct 31, 2024
fe2d008
really exclude segfaulting crates
Alex6323 Oct 31, 2024
92e4b16
bad docs
Alex6323 Oct 31, 2024
5630216
disable swap
Alex6323 Oct 31, 2024
9d6c8db
ensure report creation
Alex6323 Oct 31, 2024
c2ed873
force success if report was generated
Alex6323 Oct 31, 2024
c88c669
clean up the mess
Alex6323 Oct 31, 2024
cc8f394
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Oct 31, 2024
bd14b7b
remove grcov coverage workflow
Alex6323 Oct 31, 2024
136cb52
more clean up
Alex6323 Oct 31, 2024
9064731
nit
Alex6323 Oct 31, 2024
a18c799
merge nextest and simtest coverage
Alex6323 Oct 31, 2024
f47ef7a
add llvm-cov dir
Alex6323 Oct 31, 2024
44f04ac
remove possibly corrupted .profraw files before report generation
Alex6323 Nov 4, 2024
af8dc1c
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Nov 4, 2024
ed1b9f6
use llvm-profdata tool installed by rustup
Alex6323 Nov 4, 2024
3ae990d
report subcommand
Alex6323 Nov 4, 2024
ebee880
check llvm-profdata version
Alex6323 Nov 4, 2024
2d7005a
nightly please
Alex6323 Nov 4, 2024
6f3b9b4
.
Alex6323 Nov 4, 2024
08ed5d8
bash fun
Alex6323 Nov 4, 2024
f72dbc2
funny commits
Alex6323 Nov 4, 2024
20c4296
hopefully fixed
Alex6323 Nov 4, 2024
19ad03f
more debug, increase retention
Alex6323 Nov 4, 2024
40d45e5
ensure correct toolchain
Alex6323 Nov 4, 2024
7f8e79d
clean up
Alex6323 Nov 4, 2024
1899874
fix output dir
Alex6323 Nov 4, 2024
3127a0d
clean llvm-cov-target
Alex6323 Nov 4, 2024
a6ad392
increase workflow timeout
Alex6323 Nov 5, 2024
2273e3b
Upload code coverage report (#3896)
shufps Nov 5, 2024
a25e2c3
remove comment
Alex6323 Nov 5, 2024
fcd80a4
fix path
Alex6323 Nov 5, 2024
370a023
clean up
Alex6323 Nov 5, 2024
e4616c2
profraw-only
Alex6323 Nov 5, 2024
21ed31b
temp disable simtest coverage
Alex6323 Nov 5, 2024
105efa0
try to improve runtime speed during coverage computation
Alex6323 Nov 5, 2024
4c66b80
sudo apt-get
Alex6323 Nov 5, 2024
e5665b1
validity check notification
Alex6323 Nov 5, 2024
ecc6986
simplify
Alex6323 Nov 5, 2024
b46600a
re-enable simtest
Alex6323 Nov 6, 2024
cdcd2bf
only simtest
Alex6323 Nov 6, 2024
4548eaf
clean env vars
Alex6323 Nov 6, 2024
fd18d03
nits
Alex6323 Nov 7, 2024
938e483
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Nov 7, 2024
c00dfa5
only code-coverage for workspace crates
Alex6323 Nov 7, 2024
380b69e
optimize
Alex6323 Nov 7, 2024
0f79efb
ignore external crates source files
Alex6323 Nov 7, 2024
7f2e147
put option on the right command
Alex6323 Nov 7, 2024
266edcc
simtests only: no debug info
Alex6323 Nov 8, 2024
ef977b9
debug log
Alex6323 Nov 8, 2024
2a70d43
re-enable nextest
Alex6323 Nov 8, 2024
44fcdf1
re-enable debug symbols
Alex6323 Nov 9, 2024
ad11ce2
reduce opt-level
Alex6323 Nov 9, 2024
d96e682
reduce opt-level to 0
Alex6323 Nov 10, 2024
f628431
refactor
Alex6323 Nov 11, 2024
19604d2
env
Alex6323 Nov 12, 2024
7e067cc
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Nov 12, 2024
0d2be43
fix env
Alex6323 Nov 12, 2024
499e94d
pass as list to llvm-profdata
Alex6323 Nov 12, 2024
9b3f251
too bad
Alex6323 Nov 12, 2024
1e41722
fix paths
Alex6323 Nov 12, 2024
2df29ea
nightly advanced infos
Alex6323 Nov 12, 2024
5836bda
fix toolchain
Alex6323 Nov 12, 2024
a10ba8c
can't have everything
Alex6323 Nov 12, 2024
b0a4422
try again
Alex6323 Nov 13, 2024
cbd12fe
test if --branch caused the OOM
Alex6323 Nov 13, 2024
8abecc7
re-include simtests
Alex6323 Nov 13, 2024
943deb0
Merge branch develop' into dev-tools/reenable-coverage-ci
Alex6323 Nov 14, 2024
855749e
disable debug symbols
Alex6323 Nov 14, 2024
809b640
single package
Alex6323 Nov 14, 2024
7181223
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Nov 20, 2024
7de2a39
disable branch coverage
Alex6323 Nov 21, 2024
09bb6b0
nit
Alex6323 Nov 21, 2024
8606253
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Nov 25, 2024
26b1423
publish to coveralls
Alex6323 Nov 25, 2024
88f8e61
set base-path
Alex6323 Nov 26, 2024
74cbc9b
upload to codecov for comparison
Alex6323 Nov 26, 2024
27e42fb
relative paths then
Alex6323 Nov 26, 2024
8fd8d17
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Nov 26, 2024
188fbc6
wen better docs?
Alex6323 Nov 26, 2024
a372302
relative paths the hard way
Alex6323 Nov 26, 2024
9b783d8
ran into coveralls issue #1530
Alex6323 Nov 26, 2024
1787c7a
upload to codecov with token
Alex6323 Nov 27, 2024
86d5224
re-enable simtest coverage :sweat: :pray:
Alex6323 Nov 27, 2024
9246ccc
don't fail if codecov upload fails
Alex6323 Nov 27, 2024
8fd0834
no nightly, fixes, cleanup, improvements
Alex6323 Nov 28, 2024
e6b3b1f
fix check
Alex6323 Nov 28, 2024
d26b3fc
come on
Alex6323 Nov 28, 2024
db168fa
sccache check
Alex6323 Nov 28, 2024
b1dc285
debug commands; use subshell
Alex6323 Nov 28, 2024
fa7db49
sccache + more echos
Alex6323 Nov 28, 2024
d0e9bab
add Coveralls badge to README.md
Alex6323 Nov 28, 2024
6e48ec6
bye codecov
Alex6323 Nov 29, 2024
9a6ad5e
fix config.toml extension
Alex6323 Nov 29, 2024
6cb1637
rm badge
Alex6323 Nov 29, 2024
8732f9f
rm sscache install; mandatory corruption check
Alex6323 Nov 29, 2024
016f593
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Nov 29, 2024
1cbe6d3
dprint
Alex6323 Nov 29, 2024
7f6e7a7
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Nov 29, 2024
d63895f
dprint Cargo.toml
Alex6323 Nov 29, 2024
5cb5452
run cargo_sort.py
Alex6323 Dec 2, 2024
c07b19c
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Dec 2, 2024
8f9135f
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Dec 2, 2024
5766415
chore(ci): clean version pin comments (#4317)
thibault-martinez Dec 3, 2024
a8a7a0f
align action pinning
Alex6323 Dec 3, 2024
04f8b17
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Dec 3, 2024
f797c08
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Dec 3, 2024
508f5bf
dprint
Alex6323 Dec 3, 2024
8d1b26a
review
Alex6323 Dec 3, 2024
88648dc
review 2
Alex6323 Dec 3, 2024
f08d266
reduce log output
Alex6323 Dec 3, 2024
7535611
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Dec 3, 2024
c8faab0
rm backing up lcov.info
Alex6323 Dec 4, 2024
2245843
Merge branch 'develop' into dev-tools/reenable-coverage-ci
Alex6323 Dec 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
176 changes: 44 additions & 132 deletions .github/workflows/cargo_llvm_cov.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
name: Code Coverage
name: Code Coverage (llvm-cov)

on:
schedule:
- cron: "0 9 * * *" # UTC timing is every day at 1am PST
- cron: "0 1 * * *" # every day at 1am (UTC)
workflow_dispatch:

env:
Expand Down Expand Up @@ -30,13 +30,17 @@ env:
RUSTDOCFLAGS: -D warnings

jobs:
cargo-llvm-cov:
llvm-cov:
name: Generate code coverage
runs-on: [self-hosted]
runs-on: self-hosted
timeout-minutes: 120
env:
CARGO_TERM_COLOR: always
steps:
# can be removed when actions are available on the image already
- name: Update Rust toolchain
run: rustup update stable

- uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # pin@v4
- uses: bmwill/rust-cache@fb63fcd7a959767755b88b5af2f5cbf65fb8a127 # pin@v1

Expand All @@ -47,139 +51,47 @@ jobs:
uses: taiki-e/install-action@7e58f89e24a544d88f7a74c6eed8a3df3fd4c658 # pin@nextest

- name: Set Swap Space
uses: pierotofy/set-swap-space@49819abfb41bd9b44fb781159c033dba90353a7c # pin@master
with:
swap-size-gb: 256

- name: Install Rust
run: rustup update stable

- name: Run code coverage for nextest
run: IOTA_SKIP_SIMTESTS=1 cargo llvm-cov --ignore-run-fail --lcov --output-path lcov.info nextest

- name: Upload report to Codecov for nextest
uses: codecov/codecov-action@b9fd7d16f6d7d1b5d2bec1a2887e65ceed900238 # pin@v4
uses: actionhippie/swap-space@v1
with:
files: lcov.info
size: 256G

DaughterOfMars marked this conversation as resolved.
Show resolved Hide resolved
- name: Run code coverage for simtest
- name: Generate code coverage (nextest)
run: |
git clean -fd
./scripts/simtest/codecov.sh

- name: Upload report to Codecov for simtest
uses: codecov/codecov-action@b9fd7d16f6d7d1b5d2bec1a2887e65ceed900238 # pin@v4
with:
files: lcov-simtest.info

notify:
name: Notify
needs: [cargo-llvm-cov]
runs-on: self-hosted
if: always() # always notify

steps:
- uses: technote-space/workflow-conclusion-action@45ce8e0eb155657ab8ccf346ade734257fd196a5 # pin@v3

- name: Checkout iota repo develop branch
uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # pin@v4

- name: Get iota commit
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
export iota_sha=$(git rev-parse HEAD)
echo "iota_sha=${iota_sha}" >> $GITHUB_ENV

- name: Get a branch name for a iota commit
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
export iota_branch_name=$(gh api -H 'Accept: application/vnd.github+json' /repos/iotaledger/iota/commits/${{ env.iota_sha }}/branches-where-head --jq '.[].name' | head -n 1)
# if the commit is not the head of the branch, get it's base branch
[[ -z $iota_branch_name ]] && export iota_branch_name=$(gh api -H 'Accept: application/vnd.github+json' /repos/iotaledger/iota/commits/${{ env.iota_sha }}/pulls --jq '.[].base.ref' | head -n 1)
echo "iota_branch_name=${iota_branch_name}" >> $GITHUB_ENV
echo "iota_branch_name_url=$(echo ${iota_branch_name} | sed 's\/\%2F\g')" >> $GITHUB_ENV

- name: Get link to logs
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
cargo llvm-cov clean
set +e
IOTA_SKIP_SIMTESTS=1 cargo +nightly llvm-cov --ignore-run-fail --branch --lcov --output-path target/llvm-cov/nextest.lcov nextest
if [ -f "target/llvm-cov/nextest.lcov" ]; then
exit 0
else
echo "Fatal Error: Failed to create nextest coverage report."
exit 1
fi

- name: Generate code coverage (simtest)
run: |
gh_job_link=$(gh api -X GET 'repos/iotaledger/iota/actions/runs/${{ github.run_id }}/jobs' --jq '.jobs.[0].html_url')
echo "gh_job_link=${gh_job_link}" >> $GITHUB_ENV

- name: Get current oncall
git clean -fd
cargo llvm-cov clean
set +e
./scripts/simtest/simtest-cov.sh
if [ -f "target/llvm-cov/simtest.lcov" ]; then
exit 0
else
echo "Fatal Error: Failed to create simtest coverage report."
exit 1
fi

- name: Create report
run: |
export current_oncall=$(curl -s --request GET \
--url 'https://api.pagerduty.com/oncalls?schedule_ids[]=PGCQ3YS' \
--header 'Accept: application/json' \
--header 'Authorization: Token token=${{ secrets.PAGERDUTY_ACCESS_KEY }}' \
--header 'Content-Type: application/json' \
| jq '.oncalls[].user.summary' | tr -d '"')
echo "current_oncall=$(echo ${current_oncall})" >> $GITHUB_ENV

export oncall_name=$(curl -s --request GET \
--url 'https://api.pagerduty.com/oncalls?schedule_ids[]=PGCQ3YS' \
--header 'Accept: application/json' \
--header 'Authorization: Token token=${{ secrets.PAGERDUTY_ACCESS_KEY }}' \
--header 'Content-Type: application/json' \
| jq '.oncalls[].escalation_policy.summary' | tr -d '"')
echo "oncall_name=$(echo ${oncall_name})" >> $GITHUB_ENV
apt install -yy lcov
genhtml -o target/llvm-cov/coverage_report --branch-coverage target/llvm-cov/nextest.lcov target/llvm-cov/simtest.lcov

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # pin@v4
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2

- name: Get slack id for the oncall
run: |
export slack_id=$(aws s3 cp s3://iota-employees-dir/employees.json - | jq --arg ONCALL "${{ env.current_oncall }}" '.[] | if .name == $ONCALL then .slack_id else empty end')
echo "slack_id=$(echo ${slack_id} | tr -d '"')" >> $GITHUB_ENV
- name: Pack report
run: cd target/llvm-cov && tar -czvf coverage_report.tgz ./coverage_report

- name: Post to slack
uses: slackapi/slack-github-action@936158bbe252e9a6062e793ea4609642c966e302 # pin@v1.21.0
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
IOTA_SHA: ${{ env.iota_sha }}
IOTA_BRANCH_NAME: ${{ env.iota_branch_name }}
IOTA_BRANCH_NAME_URL: ${{ env.iota_branch_name_url }}
GH_JOB_LINK: ${{ env.gh_job_link }}
SLACK_ID: ${{ env.slack_id }}
ONCALL_NAME: ${{ env.oncall_name }}
- name: Upload report
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # pin@v4
with:
thibault-martinez marked this conversation as resolved.
Show resolved Hide resolved
channel-id: "code-coverage"
payload: |
{
"text": "*${{ github.workflow }}* workflow status: `${{ env.WORKFLOW_CONCLUSION }}`",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*${{ github.workflow }}* workflow status: `${{ env.WORKFLOW_CONCLUSION }}`"
}
},
{
"type": "divider"
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "IOTA commit: <https://github.com/iotaledger/iota/commit/${{ env.IOTA_SHA }}|${{ env.IOTA_SHA }}>\nIOTA branch: `${{ env.IOTA_BRANCH_NAME }}`\nRun: <${{ env.GH_JOB_LINK }}|${{ github.run_id }}>"
}
},
{
"type": "divider"
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "<@${{ env.SLACK_ID }}>, current `${{ env.ONCALL_NAME }}` oncall, please look over the code coverage <https://app.codecov.io/github/iotaledger/iota/tree/${{ env.IOTA_BRANCH_NAME_URL }}|report> for the `${{ env.IOTA_BRANCH_NAME }}` branch in IOTA repo."
}
}
]
}
name: coverage_report.tgz
path: ./target/llvm-cov/coverage_report.tgz
if-no-files-found: error
retention-days: 1
64 changes: 0 additions & 64 deletions .github/workflows/codecov.yml

This file was deleted.

29 changes: 15 additions & 14 deletions scripts/simtest/config-patch
thibault-martinez marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -1,22 +1,23 @@
diff --git a/.cargo/config b/.cargo/config
index ec2c459490..55985cbe9f 100644
--- a/.cargo/config
+++ b/.cargo/config
@@ -25,4 +25,4 @@ move-clippy = [
]

diff --git a/.cargo/config.toml b/.cargo/config.toml
index cc5620ecfe..d2ad8f1dac 100644
--- a/.cargo/config.toml
+++ b/.cargo/config.toml
@@ -6,7 +6,7 @@ ci-udeps = "udeps --all-targets --backend=depinfo"
ci-udeps-external = "udeps --all-targets --manifest-path external-crates/move/Cargo.toml --backend=depinfo"
[build]
-rustflags = ["-C", "force-frame-pointers=yes", "-C", "force-unwind-tables=yes"]
+rustflags = ["-C", "force-frame-pointers=yes", "-C", "force-unwind-tables=yes", "--cfg", "msim"]

# 64 bit MSVC, override default 1M stack with 8M stack
[target.x86_64-pc-windows-msvc]
diff --git a/Cargo.toml b/Cargo.toml
index c0829bc1b6..4007f97d66 100644
index 6fd5380358..f2111d330c 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -682,5 +682,7 @@ field_names = "0.2.0"
semver = "1.0.16"
spinners = "4.1.0"
include_dir = "0.7.3"
+
+[patch.crates-io]
@@ -477,3 +477,5 @@ typed-store-derive = { path = "crates/typed-store-derive" }
typed-store-error = { path = "crates/typed-store-error" }

[patch.crates-io]
+tokio = { git = "ssh://git@github.com/iotaledger/iota-sim.git", rev = "8fe1939f4bf1e54d0638756c82aaa6c29212e2bb" }
+futures-timer = { git = "ssh://git@github.com/iotaledger/iota-sim.git", rev = "8fe1939f4bf1e54d0638756c82aaa6c29212e2bb" }
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
# verify that git repo is clean
if [[ -n $(git status -s) ]]; then
echo "Working directory is not clean. Please commit all changes before running this script."
echo $(git status -s)
exit 1
fi

Expand All @@ -15,7 +16,8 @@ git apply ./scripts/simtest/config-patch
root_dir=$(git rev-parse --show-toplevel)
export SIMTEST_STATIC_INIT_MOVE=$root_dir"/examples/move/basics"

MSIM_WATCHDOG_TIMEOUT_MS=60000 MSIM_TEST_SEED=1 cargo llvm-cov --ignore-run-fail --lcov --output-path lcov-simtest.info nextest --cargo-profile simulator
MSIM_WATCHDOG_TIMEOUT_MS=60000 MSIM_TEST_SEED=1 cargo +nightly llvm-cov --ignore-run-fail --branch --lcov --output-path target/llvm-cov/simtest.lcov \
nextest --cargo-profile simulator

# remove the patch
git checkout .cargo/config Cargo.toml Cargo.lock
Alex6323 marked this conversation as resolved.
Show resolved Hide resolved
Loading