adder-collator: add velocity measurement and make elastic scaling test more robust

[sandreim]

Improves adder-collator to also compute the parachain velocity. The velocity is defined as number of parachain blocks progressing per relay chain block.

In this test we're asserting that the elastic parachain always progresses by 3 blocks per RCB, while the non-elastic parachain progresses normally - 1 block per RCB.

[sandreim]

Passed on my machine, hope it passes in CI

[ordian]

The test passed on the first try, so looks good.

[ordian]

Spoke too soon? 🙈 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5821508

[bkchr]

The test passed on the first try, so looks good.

Flaky tests have the property of doing so :P

[ordian]

I'd argue that in this test what we actually want to measure depends on the hardware not being flaky/potato, which it is not in CI :P

[sandreim]

damn, but I wonder how much we overcommit on these CI instances, maybe we should run this on with benchmark tag.

[sandreim]

@pepoviola do we have any way of reducing overcommit on the cluster instances for some tests ?

[sandreim]

Looking at the test resources we have for relay chain:

  limits = { memory = "4G", cpu = "2" }
  requests = { memory = "2G", cpu = "1" }

Does this work for collators ? And do we actually reserve these resources vs other work ?

[pepoviola]

Looking at the test resources we have for relay chain:

  limits = { memory = "4G", cpu = "2" }
  requests = { memory = "2G", cpu = "1" }

Does this work for collators ? And do we actually reserve these resources vs other work ?

Yes, this will work for collators too. As reference we use vms with 8 vcpus/16G of ram. So, you can increase the request to place 2 collators per vm (or even one but let me know to notice in the budget). Ping me if I can help you with something else.

Thanks!!

[bkchr]

maybe we should run this on with benchmark tag

I had the same idea. Ser @sandreim, can we not just run zombienet with the local provider on the benchmarking machine? :D

[pepoviola]

maybe we should run this on with benchmark tag

I had the same idea. Ser @sandreim, can we not just run zombienet with the local provider on the benchmarking machine? :D

We can add the run-native version but we need to wire all the monitoring stack (that is already there with kubernetes). Other option is to add a label in the config for the nodes that needs to run in an isolated vm to get better performance and that way we can have the granularity to set one vm per node if needed.

Thx!

[sandreim]

maybe we should run this on with benchmark tag

I had the same idea. Ser @sandreim, can we not just run zombienet with the local provider on the benchmarking machine? :D

We could do that, but as @pepoviola is saying we would have trouble getting the logs when a test fails. I have hopes in a new label for pods or just better k8s scheduling of pods such that the claimed resources are always guaranteed by it. Javier is looking into this right now.