Cherry-pick VM changes #12021

vgao1996 · 2024-02-14T22:17:05Z

This cherry picks a few VM changes we have been working on recently.

trunk-io · 2024-02-14T22:17:09Z

⏱️ 40h 15m total CI duration on this PR

Job	Cumulative Duration	Recent Runs
rust-unit-tests	8h 50m	⬜ 🟩 🟥 🟥 🟩 (+3 more)
rust-unit-coverage	5h 32m	🟩
rust-move-unit-coverage	3h 54m	🟥 🟩 🟩 🟩 🟩 (+2 more)
rust-smoke-coverage	3h 50m	🟩
execution-performance / single-node-performance	3h 45m	🟥 🟥 🟥 🟩 🟩 (+4 more)
windows-build	2h 49m	🟩 🟩 🟩 🟩 🟩 (+4 more)
rust-smoke-tests	2h 35m	⬜ 🟩 🟩 🟩 🟩 (+2 more)
rust-move-tests	1h 52m	🟥 🟩 🟩 🟩 🟩 (+2 more)
forge-e2e-test / forge	1h 25m	🟥 🟥 🟩 🟩 🟩 (+1 more)
forge-compat-test / forge	1h 8m	🟩 🟩 🟩 🟩 🟩
rust-images / rust-all	54m	⬜ 🟩 🟩 🟩 🟩 (+2 more)
cli-e2e-tests / run-cli-tests	52m	🟥 🟥 🟥 🟥 🟥 (+1 more)
rust-lints	48m	⬜ 🟩 🟩 🟩 🟩 (+3 more)
run-tests-main-branch	30m	⬜ 🟥 🟥 🟥 🟥 (+3 more)
check	30m	🟩 🟩 🟩 🟩 🟩 (+4 more)
general-lints	21m	🟩 🟩 🟩 🟩 🟩 (+3 more)
check-dynamic-deps	18m	🟩 🟩 🟩 🟩 🟩 (+4 more)
indexer-grpc-e2e-tests / test-indexer-grpc-docker-compose	6m	🟩 🟩 🟥 🟥
node-api-compatibility-tests / node-api-compatibility-tests	4m	🟩 🟩 🟩 🟩 🟩
semgrep/ci	3m	🟩 🟩 🟩 🟩 🟩 (+4 more)
file_change_determinator	1m	🟩 🟩 🟩 🟩 🟩 (+2 more)
file_change_determinator	1m	🟩 🟩 🟩 🟩 🟩 (+4 more)
file_change_determinator	1m	🟩 🟩 🟩 🟩 🟩 (+3 more)
execution-performance / file_change_determinator	1m	🟩 🟩 🟩 🟩 🟩 (+2 more)
permission-check	35s	🟩 🟩 🟩 🟩 🟩 (+4 more)
permission-check	33s	🟩 🟩 🟩 🟩 🟩 (+4 more)
permission-check	32s	🟩 🟩 🟩 🟩 🟩 (+4 more)
permission-check	28s	🟩 🟩 🟩 🟩 🟩 (+4 more)
permission-check	22s	🟩 🟩 🟩 🟩 🟩 (+2 more)
upload-to-codecov	15s	🟩
determine-docker-build-metadata	12s	🟩 🟩 🟩 🟩 🟩 (+2 more)

🚨 2 jobs on the last run were significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
cli-e2e-tests / run-cli-tests	9m	6m
windows-build	12m	18m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

codecov · 2024-02-14T22:58:18Z

Codecov Report

Attention: 9 lines in your changes are missing coverage. Please review.

Comparison is base (f86d996) 70.2% compared to head (1f226a1) 71.5%.
Report is 7 commits behind head on main.

❗ Current head 1f226a1 differs from pull request most recent head 2baf8db. Consider uploading reports for the commit 2baf8db to get more accurate results

Files	Patch %	Lines
...ove/move-vm/types/src/loaded_data/runtime_types.rs	94.7%	6 Missing ⚠️
aptos-move/aptos-gas-meter/src/meter.rs	88.8%	1 Missing ⚠️
...aptos-gas-schedule/src/gas_schedule/transaction.rs	0.0%	1 Missing ⚠️
third_party/move/move-vm/runtime/src/loader/mod.rs	96.0%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main   #12021       +/-   ##
===========================================
+ Coverage    70.2%    71.5%     +1.2%     
===========================================
  Files        2220      802     -1418     
  Lines      418896   184339   -234557     
===========================================
- Hits       294344   131804   -162540     
+ Misses     124552    52535    -72017

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…id mode

ziaptos

Linter says eof newline missing

… paranoid mode

github-actions · 2024-02-15T18:24:51Z

✅ Forge suite `compat` success on `aptos-node-v1.8.3` ==> `2baf8db5e1cd85c2fbf51e15f998aab2743e6f4f`

Compatibility test results for aptos-node-v1.8.3 ==> 2baf8db5e1cd85c2fbf51e15f998aab2743e6f4f (PR)
1. Check liveness of validators at old version: aptos-node-v1.8.3
compatibility::simple-validator-upgrade::liveness-check : committed: 4824 txn/s, latency: 6584 ms, (p50: 6300 ms, p90: 9300 ms, p99: 14200 ms), latency samples: 178500
2. Upgrading first Validator to new version: 2baf8db5e1cd85c2fbf51e15f998aab2743e6f4f
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1813 txn/s, latency: 15493 ms, (p50: 19300 ms, p90: 21600 ms, p99: 22000 ms), latency samples: 92480
3. Upgrading rest of first batch to new version: 2baf8db5e1cd85c2fbf51e15f998aab2743e6f4f
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1838 txn/s, latency: 15965 ms, (p50: 18900 ms, p90: 22200 ms, p99: 22300 ms), latency samples: 93740
4. upgrading second batch to new version: 2baf8db5e1cd85c2fbf51e15f998aab2743e6f4f
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3146 txn/s, latency: 9646 ms, (p50: 9600 ms, p90: 17300 ms, p99: 18100 ms), latency samples: 138460
5. check swarm health
Compatibility test for aptos-node-v1.8.3 ==> 2baf8db5e1cd85c2fbf51e15f998aab2743e6f4f passed
Test Ok

github-actions · 2024-02-15T18:27:43Z

✅ Forge suite `realistic_env_max_load` success on `2baf8db5e1cd85c2fbf51e15f998aab2743e6f4f`

two traffics test: inner traffic : committed: 7480 txn/s, latency: 5095 ms, (p50: 4500 ms, p90: 6300 ms, p99: 14700 ms), latency samples: 3231780
two traffics test : committed: 100 txn/s, latency: 2373 ms, (p50: 2200 ms, p90: 2500 ms, p99: 8400 ms), latency samples: 1880
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.285, avg: 0.221", "QsPosToProposal: max: 0.178, avg: 0.144", "ConsensusProposalToOrdered: max: 0.569, avg: 0.517", "ConsensusOrderedToCommit: max: 0.488, avg: 0.463", "ConsensusProposalToCommit: max: 0.995, avg: 0.980"]
Max round gap was 2 [limit 4] at version 765553. Max no progress secs was 9.844295 [limit 15] at version 765553.
Test Ok

* clean error log lines (#12019) * [table info][2/4] add utils for table info backup and restore and redesign the db read (#11793) * separate indexer async v2 db from aptosdb * address comments * add utils for table info backup and restore and redesign the db read * address comments to spawn block sync file ops * address comments * tests for events and improve event v1 handling (#12012) * [move-vm] Cache verified modules (#12002) * [move-vm] Cache verified modules * fixup! [move-vm] Cache verified modules * [passkey] Add MAX_BYTES limit for signatures (#11697) * [passkey] Add MAX_BYTES limit for signatures * [passkey] Add tracing for AssertionSignature type and fix README * [passkey] Rebased on latest main, rerun authenticator_regenerate.sh * Object Code Deployment module with CLI commands generated (#11748) * [simple] rename RG split in VmChangeSet flag (#12027) * rename RG split in VmChangeSet flag old name was stale, when charging was different * [fuzzing] fixes oss-fuzz FP and fuzz.sh (#12030) * [fuzzing] fixes oss-fuzz FP and fuzz.sh * Update Docker images (#12026) Co-authored-by: sionescu <sionescu@users.noreply.github.com> * Update release.yaml (#12020) * Update release.yaml * enable REFUNDABLE_BYTES * enable FairnessShuffler * enable WEBAUTHN_SIGNATURE * AIP-54 Object Code Deployment release addition * enable vtxn and jwk consensus * Update release.yaml adding aggregators v2 flags, and updating execution onchain config * add feature flag for zkID (ZK-only mode) * fix jwk/zkid entries in release yaml 1.10 (#12024) * update * update * Update release.yaml fix flag name * Update release.yaml rename feature --------- Co-authored-by: aldenhu <msmouse@gmail.com> Co-authored-by: hariria <hariria@usc.edu> Co-authored-by: John Chang <johnchang9094@gmail.com> Co-authored-by: danielxiangzl <xiangzhuolun@gmail.com> Co-authored-by: igor-aptos <110557261+igor-aptos@users.noreply.github.com> Co-authored-by: Alin Tomescu <tomescu.alin@gmail.com> Co-authored-by: zhoujunma <zjma@users.noreply.github.com> * Cherry-pick VM changes (#12021) * [gas] add gas charges for type creation * [gas-calibration] Add calibration sample * [move-vm] Implement a per-frame cache for paranoid mode * fixup! [move-vm] Implement a per-frame cache for paranoid mode * fixup! fixup! [move-vm] Implement a per-frame cache for paranoid mode * fixup! fixup! fixup! [move-vm] Implement a per-frame cache for paranoid mode * fixup! fixup! fixup! fixup! [move-vm] Implement a per-frame cache for paranoid mode * [gas] add gas charges for dependencies --------- Co-authored-by: Runtian Zhou <runtian@aptoslabs.com> * trivial doc fix * [GHA] Upgrade actions/checkout to v4 * jwk ob counters (#12048) * Revert "[GHA] Upgrade actions/checkout to v4" This reverts commit 04d078f. * [CI][indexer] fix the e2e localnet. (#12047) * fix the e2e localnet. * fix the e2e localnet. * bump latest gas feature version to 14 Also be conservative and leave legacy parameters in >14 versions for now. Need to clean up after REFUNDABLE_BYTES feature is actually enabled on all networks. * compat test to be against the testnet tag * [GHA] Upgrade lint-test.yaml and the dependent actions to checkout@v4 actions/checkout@v4 doesn't behave well if both a workflow and an invoked action checkout the source code on top of each other. * [GHA] Update pin for tj-actions/changed-files * start jwk consensus for google (#12053) * [consensus] check rpc epoch in epoch_manager (#12018) * [consensus] check rpc epoch in epoch_manager * fix gas version (13 is deprecated/cannot be used) (#12064) * FatalVMError shouldn't create "Delayed materialization code invariant" (#12044) * Move all visibility checking into AST-level function_checker, simplify that code a bit, and improve diagnostics. (#11948) * rust changes to move all visibility checking to AST and clean it up a bit * change `Known attribute ... position` warning to a neater `Attribute .. position` warning * add FunctionData id_loc to allow pointing at function name in declaration for more concise error messages. abstract messages a bit in function_checker * add 'inlined from' labels to diagnostics with labels, fix bug in function_checker to enable post-inlining visibility checking * lint * fix for small stakes * assert --------- Co-authored-by: igor-aptos <110557261+igor-aptos@users.noreply.github.com> Co-authored-by: jill <121921928+jillxuu@users.noreply.github.com> Co-authored-by: George Mitenkov <georgemitenk0v@gmail.com> Co-authored-by: runtianz <runtian@aptoslabs.com> Co-authored-by: Andrew Hariri <hariria@usc.edu> Co-authored-by: John Chang <johnchang9094@gmail.com> Co-authored-by: Gerardo Di Giacomo <gerardo@aptoslabs.com> Co-authored-by: sionescu <sionescu@users.noreply.github.com> Co-authored-by: Junkil Park <jpark@aptoslabs.com> Co-authored-by: aldenhu <msmouse@gmail.com> Co-authored-by: danielxiangzl <xiangzhuolun@gmail.com> Co-authored-by: Alin Tomescu <tomescu.alin@gmail.com> Co-authored-by: Victor Gao <10379359+vgao1996@users.noreply.github.com> Co-authored-by: Stelian Ionescu <stelian@aptoslabs.com> Co-authored-by: Stelian Ionescu <sionescu@cddr.org> Co-authored-by: larry-aptos <112209412+larry-aptos@users.noreply.github.com> Co-authored-by: Balaji Arun <balaji@aptoslabs.com> Co-authored-by: Brian R. Murphy <132495859+brmataptos@users.noreply.github.com>

vgao1996 requested a review from runtian-zhou February 14, 2024 22:17

vgao1996 requested review from davidiw, wrwg, zekun000, georgemitenkov, banool, gregnazario and 0xmaayan as code owners February 14, 2024 22:17

runtian-zhou approved these changes Feb 14, 2024

View reviewed changes

vgao1996 requested review from ziaptos and igor-aptos February 15, 2024 00:02

igor-aptos approved these changes Feb 15, 2024

View reviewed changes

vgao1996 force-pushed the cp-1-10 branch from 0f042ba to a2cecd6 Compare February 15, 2024 00:04

vgao1996 enabled auto-merge (squash) February 15, 2024 00:04

This comment has been minimized.

Sign in to view

vgao1996 and others added 6 commits February 15, 2024 01:50

[gas] add gas charges for type creation

6c195ba

[gas-calibration] Add calibration sample

8017dc0

[move-vm] Implement a per-frame cache for paranoid mode

5811bce

fixup! [move-vm] Implement a per-frame cache for paranoid mode

c4fb108

fixup! fixup! [move-vm] Implement a per-frame cache for paranoid mode

3af5b88

fixup! fixup! fixup! [move-vm] Implement a per-frame cache for parano…

681c259

…id mode

vgao1996 force-pushed the cp-1-10 branch from a2cecd6 to 0adb593 Compare February 15, 2024 01:51

ziaptos approved these changes Feb 15, 2024

View reviewed changes

vgao1996 force-pushed the cp-1-10 branch from 0adb593 to e0dff68 Compare February 15, 2024 02:09

vgao1996 added the CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR label Feb 15, 2024

This comment has been minimized.

Sign in to view

vgao1996 force-pushed the cp-1-10 branch from e0dff68 to 52e91cb Compare February 15, 2024 03:33

This comment has been minimized.

Sign in to view

vgao1996 force-pushed the cp-1-10 branch from 52e91cb to 1f226a1 Compare February 15, 2024 17:17

This comment has been minimized.

Sign in to view

fixup! fixup! fixup! fixup! [move-vm] Implement a per-frame cache for…

2baf8db

… paranoid mode

vgao1996 force-pushed the cp-1-10 branch from 1f226a1 to 2baf8db Compare February 15, 2024 18:00

This comment has been minimized.

Sign in to view

vgao1996 merged commit 6b16e80 into aptos-labs:main Feb 15, 2024
43 of 45 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cherry-pick VM changes #12021

Cherry-pick VM changes #12021

vgao1996 commented Feb 14, 2024

trunk-io bot commented Feb 14, 2024 •

edited

Loading

codecov bot commented Feb 14, 2024 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

ziaptos left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Feb 15, 2024

github-actions bot commented Feb 15, 2024

Cherry-pick VM changes #12021

Cherry-pick VM changes #12021

Conversation

vgao1996 commented Feb 14, 2024

trunk-io bot commented Feb 14, 2024 • edited Loading

codecov bot commented Feb 14, 2024 • edited Loading

Codecov Report

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

ziaptos left a comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Feb 15, 2024

✅ Forge suite compat success on aptos-node-v1.8.3 ==> 2baf8db5e1cd85c2fbf51e15f998aab2743e6f4f

github-actions bot commented Feb 15, 2024

✅ Forge suite realistic_env_max_load success on 2baf8db5e1cd85c2fbf51e15f998aab2743e6f4f

trunk-io bot commented Feb 14, 2024 •

edited

Loading

codecov bot commented Feb 14, 2024 •

edited

Loading

✅ Forge suite `compat` success on `aptos-node-v1.8.3` ==> `2baf8db5e1cd85c2fbf51e15f998aab2743e6f4f`

✅ Forge suite `realistic_env_max_load` success on `2baf8db5e1cd85c2fbf51e15f998aab2743e6f4f`