Process PeerViewChange messages with priority
#577
Assignees
Labels
I9-optimisation
An enhancement to provide better overall performance in terms of time-to-completion for a task.
Comments
26 tasks
the-right-joyce
added
I9-optimisation
github-merge-queue bot
pushed a commit
that referenced
this issue
Jul 16, 2024
Closes #577 ### Changed - `orchestra` updated to 0.4.0 - `PeerViewChange` sent with high priority and should be processed first in a queue. - To count them in tests added tracker to TestSender and TestOverseer. It acts more like a smoke test though. ### Testing on Versi The changes were tested on Versi with two objectives: 1. Make sure the node functionality does not change. 2. See how the changes affect performance. Test setup: - 2.5 hours for each case - 100 validators - 50 parachains - validatorsPerCore = 2 - neededApprovals = 100 - nDelayTranches = 89 - relayVrfModuloSamples = 50 During the test period, all nodes ran without any crashes, which satisfies the first objective. To estimate the change in performance we used ToF charts. The graphs show that there are no spikes in the top as before. This proves that our hypothesis is correct. ### Normalized charts with ToF  [Before](https://grafana.teleport.parity.io/goto/ZoR53ClSg?orgId=1)  [After](https://grafana.teleport.parity.io/goto/6ux5qC_IR?orgId=1) ### Conclusion The prioritization of subsystem messages reduces the ToF of the networking subsystem, which helps faster propagation of gossip messages.
github-project-automation
bot
moved this from Review in progress
to Completed
in parachains team board
jpserrat
pushed a commit
to jpserrat/polkadot-sdk
that referenced
this issue
Jul 18, 2024
Closes paritytech#577 ### Changed - `orchestra` updated to 0.4.0 - `PeerViewChange` sent with high priority and should be processed first in a queue. - To count them in tests added tracker to TestSender and TestOverseer. It acts more like a smoke test though. ### Testing on Versi The changes were tested on Versi with two objectives: 1. Make sure the node functionality does not change. 2. See how the changes affect performance. Test setup: - 2.5 hours for each case - 100 validators - 50 parachains - validatorsPerCore = 2 - neededApprovals = 100 - nDelayTranches = 89 - relayVrfModuloSamples = 50 During the test period, all nodes ran without any crashes, which satisfies the first objective. To estimate the change in performance we used ToF charts. The graphs show that there are no spikes in the top as before. This proves that our hypothesis is correct. ### Normalized charts with ToF  [Before](https://grafana.teleport.parity.io/goto/ZoR53ClSg?orgId=1)  [After](https://grafana.teleport.parity.io/goto/6ux5qC_IR?orgId=1) ### Conclusion The prioritization of subsystem messages reduces the ToF of the networking subsystem, which helps faster propagation of gossip messages.
TarekkMA
pushed a commit
to moonbeam-foundation/polkadot-sdk
that referenced
this issue
Aug 2, 2024
Closes paritytech#577 ### Changed - `orchestra` updated to 0.4.0 - `PeerViewChange` sent with high priority and should be processed first in a queue. - To count them in tests added tracker to TestSender and TestOverseer. It acts more like a smoke test though. ### Testing on Versi The changes were tested on Versi with two objectives: 1. Make sure the node functionality does not change. 2. See how the changes affect performance. Test setup: - 2.5 hours for each case - 100 validators - 50 parachains - validatorsPerCore = 2 - neededApprovals = 100 - nDelayTranches = 89 - relayVrfModuloSamples = 50 During the test period, all nodes ran without any crashes, which satisfies the first objective. To estimate the change in performance we used ToF charts. The graphs show that there are no spikes in the top as before. This proves that our hypothesis is correct. ### Normalized charts with ToF  [Before](https://grafana.teleport.parity.io/goto/ZoR53ClSg?orgId=1)  [After](https://grafana.teleport.parity.io/goto/6ux5qC_IR?orgId=1) ### Conclusion The prioritization of subsystem messages reduces the ToF of the networking subsystem, which helps faster propagation of gossip messages.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This will need some flavor of paritytech/orchestra#8 implemented first.
Prioritising the processing of
PeerViewChangein networking subsystems will help reducing wasteful/useless work and faster propagation of gossip messages - approvals/assignments, bitfields and statements.We know that under high load, these messages usually have a very high ToF (up to 5s) and get processed very late by the approval distribution subsystem. With this change the peer view update is processed ahead of all other messages except signals.
The text was updated successfully, but these errors were encountered: