Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Script to automate weight generation #128

Open
liamaharon opened this issue Dec 14, 2023 · 0 comments · Fixed by #406
Open

Script to automate weight generation #128

liamaharon opened this issue Dec 14, 2023 · 0 comments · Fixed by #406

Comments

@liamaharon
Copy link
Contributor

liamaharon commented Dec 14, 2023

Now that the weight generation has been documented (#127), a script should be written to automate the process.

Should take inputs like:

  1. Set of runtimes & pallets/extrinsics to bench
  2. SSH credentials for a benchmark machine
  3. Local output dir

and pallet-by-pallet exec the benchmarks on the remote machine and write the results to the specified local directory.

the command should by default 'pick up from where it left off' if it was terminated early, e.g. be tolerant to stopping/starting and not repeat benchmarks that have already been done.

@mordamax mordamax mentioned this issue Jul 26, 2024
1 task
fellowship-merge-bot bot pushed a commit that referenced this issue Aug 26, 2024
Closes #128
Closes paritytech/product-engineering#41

Adds ability to run some standard helper commands, for now only for
benchmarks and fmt (with taplo)
It will commit the result of operation back into PR with some reports
and/or links to logs

bench uses `gitrun-001` runner which corresponds to [Validator Reference
Hardware](https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#standard-hardware)

- [x] Does not require a CHANGELOG entry

<details>
<summary>Examples (screenshots):</summary>

### Benchmarks
![Google Chrome 2024-07-26 13 33
05](https://github.com/user-attachments/assets/d1d32f7e-295a-4f52-bb69-ff93a4a61493)

### Bench results with Subweight report
![Google Chrome 2024-07-26 13 35
36](https://github.com/user-attachments/assets/e907b8f2-f818-475e-9d52-fdf48a0455f8)

### /cmd [command] --clean - removing bot's and author's command runs to
clean up PR
![Google Chrome 2024-07-26 13 33
56](https://github.com/user-attachments/assets/6e21f616-b4c1-4ca5-a89a-48e64a46c625)

### /cmd [command] --help - outputs the usage instructions
![Google Chrome 2024-07-26 13 33
25](https://github.com/user-attachments/assets/73d46493-4ab1-4f9d-a5e8-094afc4f2a49)

</details>

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant