Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[aptos-transaction-benchmark] Fix benchmark regression #5711

Merged
merged 1 commit into from
Nov 29, 2022

Conversation

runtian-zhou
Copy link
Contributor

Description

Fix the benchmark regression because of feature unification. Previously benchmark runner is always executing failed transaction and thus not providing any useful information.

Test Plan

Manually ran the benchmarker and print the transaction output to make sure we all transaction are executed successfully.

Copy link
Contributor

@gelash gelash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

strong approve

@gelash gelash added the CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR label Nov 28, 2022
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions
Copy link
Contributor

✅ Forge suite land_blocking success on 922a58e3f8725efa5769b3ffd6a0fa7574d5d4a1

performance benchmark with full nodes : 6729 TPS, 5887 ms latency, 10700 ms p99 latency,(!) expired 560 out of 2873960 txns
Test Ok

@github-actions
Copy link
Contributor

✅ Forge suite compat success on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 922a58e3f8725efa5769b3ffd6a0fa7574d5d4a1

Compatibility test results for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 922a58e3f8725efa5769b3ffd6a0fa7574d5d4a1 (PR)
1. Check liveness of validators at old version: testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b
compatibility::simple-validator-upgrade::liveness-check : 7480 TPS, 5175 ms latency, 7700 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 922a58e3f8725efa5769b3ffd6a0fa7574d5d4a1
compatibility::simple-validator-upgrade::single-validator-upgrade : 4775 TPS, 8533 ms latency, 11500 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 922a58e3f8725efa5769b3ffd6a0fa7574d5d4a1
compatibility::simple-validator-upgrade::half-validator-upgrade : 4341 TPS, 9255 ms latency, 12400 ms p99 latency,no expired txns
4. upgrading second batch to new version: 922a58e3f8725efa5769b3ffd6a0fa7574d5d4a1
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6747 TPS, 5649 ms latency, 9900 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 922a58e3f8725efa5769b3ffd6a0fa7574d5d4a1 passed
Test Ok

@runtian-zhou runtian-zhou merged commit 1c60c57 into main Nov 29, 2022
@runtian-zhou runtian-zhou deleted the runtianz/fix_benchmark branch November 29, 2022 22:18
@Markuze Markuze mentioned this pull request Dec 5, 2022
areshand pushed a commit to areshand/aptos-core-1 that referenced this pull request Dec 18, 2022
@Markuze Markuze mentioned this pull request Dec 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CICD:run-e2e-tests when this label is present github actions will run all land-blocking e2e tests from the PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants