diff --git a/.github/command-screnshot.png b/.github/command-screnshot.png deleted file mode 100644 index 1451fabca8b97..0000000000000 Binary files a/.github/command-screnshot.png and /dev/null differ diff --git a/.github/commands-readme.md b/.github/commands-readme.md deleted file mode 100644 index ce4e0fd0d789c..0000000000000 --- a/.github/commands-readme.md +++ /dev/null @@ -1,276 +0,0 @@ -# Running commands - -Command bot has been migrated, it is no longer a comment parser and now it is a GitHub action that works as a [`workflow_dispatch`](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_dispatch) event. - -## How to run an action - -To run an action, you need to go to the [_actions tab_](https://github.com/paritytech/polkadot-sdk/actions) and pick the one you desire to run. - -The current available command actions are: - -- [Command FMT](https://github.com/paritytech/polkadot-sdk/actions/workflows/command-fmt.yml) -- [Command Update UI](https://github.com/paritytech/polkadot-sdk/actions/workflows/command-update-ui.yml) -- [Command Prdoc](https://github.com/paritytech/polkadot-sdk/actions/workflows/command-prdoc.yml) -- [Command Sync](https://github.com/paritytech/polkadot-sdk/actions/workflows/command-sync.yml) -- [Command Bench](https://github.com/paritytech/polkadot-sdk/actions/workflows/command-bench.yml) -- [Command Bench All](https://github.com/paritytech/polkadot-sdk/actions/workflows/command-bench-all.yml) -- [Command Bench Overhead](https://github.com/paritytech/polkadot-sdk/actions/workflows/command-bench-overhead.yml) - -You need to select the action, and click on the dropdown that says: `Run workflow`. It is located in the upper right. - -If this dropdown is not visible, you may not have permission to run the action. Contact IT for help. - -![command screenshot](command-screnshot.png) - -Each command will have the same two required values, but it could have more. - -GitHub's official documentation: [Manually running a workflow](https://docs.github.com/en/actions/using-workflows/manually-running-a-workflow) - -#### Running from CLI - -You can use [`gh cli`](https://cli.github.com/) to run the commands too. Refers to the [`gh workflow run`](https://cli.github.com/manual/gh_workflow_run) section from the documentation for more information. - -### Number of the Pull Request - -The number of the pull request. Required so the action can fetch the correct branch and comment if it fails. - -## Action configurations - -### FMT - -For FMT you only need the PR number. - -You can use the following [`gh cli`](https://cli.github.com/) inside the repo: - -```bash -gh workflow run command-fmt.yml -f pr=1000 -``` - -### Update UI - -For Update UI you only need the PR number. - -You can use the following [`gh cli`](https://cli.github.com/) inside the repo: - -```bash -gh workflow run command-update-ui.yml -f pr=1000 -``` - -### Bench - -Runs `benchmark pallet` or `benchmark overhead` against your PR and commits back updated weights. - -Posible combinations based on the `benchmark` dropdown. - -- `substrate-pallet`: Pallet Benchmark for Substrate for specific pallet - - Requires `Subcommand` to be `pallet` - - Requires `Runtime` to be `dev` - - Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$` - - Requires `Target Directory` to be `substrate` -- `polkadot-pallet`: Pallet Benchmark for Polkadot for specific pallet - - Requires `Subcommand` to be one of the following: - - `pallet` - - `xcm` - - Requires `Runtime` to be one of the following: - - `rococo` - - `westend` - - Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$` - - Requires `Target Directory` to be `polkadot` -- `cumulus-assets`: Pallet Benchmark for Cumulus assets - - Requires `Subcommand` to be one of the following: - - `pallet` - - `xcm` - - Requires `Runtime` to be one of the following: - - `asset-hub-westend` - - `asset-hub-rococo` - - Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$` - - Requires `Runtime Dir` to be `assets` - - Requires `Target Directory` to be `cumulus` -- `cumulus-collectives`: Pallet Benchmark for Cumulus collectives - - Requires `Subcommand` to be one of the following: - - `pallet` - - `xcm` - - Requires `Runtime` to be `collectives-westend` - - Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$` - - Requires `Runtime Dir` to be `collectives` - - Requires `Target Directory` to be `cumulus` -- `cumulus-coretime`: Pallet Benchmark for Cumulus coretime - - Requires `Subcommand` to be one of the following: - - `pallet` - - `xcm` - - Requires `Runtime` to be one of the following: - - `coretime-rococo` - - `coretime-westend` - - Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$` - - Requires `Runtime Dir` to be `coretime` - - Requires `Target Directory` to be `cumulus` -- `cumulus-bridge-hubs`: Pallet Benchmark for Cumulus bridge-hubs - - Requires `Subcommand` to be one of the following: - - `pallet` - - `xcm` - - Requires `Runtime` to be one of the following: - - `bridge-hub-rococo` - - `bridge-hub-westend` - - Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$` - - Requires `Runtime Dir` to be `bridge-hub` - - Requires `Target Directory` to be `cumulus` -- `cumulus-contracts`: Pallet Benchmark for Cumulus contracts - - Requires `Subcommand` to be one of the following: - - `pallet` - - `xcm` - - Requires `Runtime` to be one `contracts-rococo` - - Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$` - - Requires `Runtime Dir` to be `contracts` - - Requires `Target Directory` to be `cumulus` -- `cumulus-glutton`: Pallet Benchmark for Cumulus glutton - - Requires `Subcommand` to be `pallet` - - Requires `Runtime` to be one of the following: - - `glutton-westend` - - `glutton-westend-dev-1300` - - Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$` - - Requires `Runtime Dir` to be `glutton` - - Requires `Target Directory` to be `cumulus` -- `cumulus-starters`: Pallet Benchmark for Cumulus starters - - Requires `Subcommand` to be one of the following: - - `pallet` - - `xcm` - - Requires `Runtime` to be one of the following: - - `seedling` - - `shell` - - Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$` - - Requires `Runtime Dir` to be `starters` - - Requires `Target Directory` to be `cumulus` -- `cumulus-people`: Pallet Benchmark for Cumulus people - - Requires `Subcommand` to be one of the following: - - `pallet` - - `xcm` - - Requires `Runtime` to be one of the following: - - `people-westend` - - `people-rococo` - - Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$` - - Requires `Runtime Dir` to be `people` - - Requires `Target Directory` to be `cumulus` -- `cumulus-testing`: Pallet Benchmark for Cumulus testing - - Requires `Subcommand` to be one of the following: - - `pallet` - - `xcm` - - Requires `Runtime` to be one of the following: - - `penpal` - - `rococo-parachain` - - Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$` - - Requires `Runtime Dir` to be `testing` - - Requires `Target Directory` to be `cumulus` - -You can use the following [`gh cli`](https://cli.github.com/) inside the repo: - -```bash -gh workflow run command-bench.yml -f pr=1000 -f benchmark=polkadot-pallet -f subcommand=pallet -f runtime=rococo -f pallet=pallet_name -f target_dir=polkadot -``` - -### Bench-all - -This is a wrapper to run `bench` for all pallets. - -Posible combinations based on the `benchmark` dropdown. - -- `pallet`: Benchmark for Substrate/Polkadot/Cumulus/Trappist for specific pallet - - Requires field `Pallet` to have an input that applies to `^([a-z_]+)([:]{2}[a-z_]+)?$` -- `substrate`: Pallet + Overhead + Machine Benchmark for Substrate for all pallets - - Requires `Target Directory` to be `substrate` -- `polkadot`: Pallet + Overhead Benchmark for Polkadot - - Requires `Runtime` to be one of the following: - - `rococo` - - `westend` - - Requires `Target Directory` to be `polkadot` -- `cumulus`: Pallet Benchmark for Cumulus - - Requires `Runtime` to be one of the following: - - `rococo` - - `westend` - - `asset-hub-kusama` - - `asset-hub-polkadot` - - `asset-hub-rococo` - - `asset-hub-westend` - - `bridge-hub-kusama` - - `bridge-hub-polkadot` - - `bridge-hub-rococo` - - `bridge-hub-westend` - - `collectives-polkadot` - - `collectives-westend` - - `coretime-rococo` - - `coretime-westend` - - `contracts-rococo` - - `glutton-kusama` - - `glutton-westend` - - `people-rococo` - - `people-westend` - - Requires `Target Directory` to be `cumulus` - -You can use the following [`gh cli`](https://cli.github.com/) inside the repo: - -```bash -gh workflow run command-bench-all.yml -f pr=1000 -f benchmark=pallet -f pallet=pallet_name -f target_dir=polkadot -f runtime=rococo -``` - -### Bench-overhead - -Run benchmarks overhead and commit back results to PR. - -Posible combinations based on the `benchmark` dropdown. - -- `default`: Runs `benchmark overhead` and commits back to PR the updated `extrinsic_weights.rs` files - - Requires `Runtime` to be one of the following: - - `rococo` - - `westend` - - Requires `Target directory` to be `polkadot` -- `substrate`: Runs `benchmark overhead` and commits back to PR the updated `extrinsic_weights.rs` files - - Requires `Target directory` to be `substrate` -- `cumulus`: Runs `benchmark overhead` and commits back to PR the updated `extrinsic_weights.rs` files - - Requires `Runtime` to be one of the following: - - `asset-hub-rococo` - - `asset-hub-westend` - - Requires `Target directory` to be `cumulus` - -You can use the following [`gh cli`](https://cli.github.com/) inside the repo: - -```bash -gh workflow run command-bench-overheard.yml -f pr=1000 -f benchmark=substrate -f runtime=rococo -f target_dir=substrate -``` - -### PrDoc - -Generate a PrDoc with the crates populated by all modified crates. - -Options: -- `pr`: The PR number to generate the PrDoc for. -- `audience`: The audience of whom the changes may concern. -- `bump`: A default bump level for all crates. The PrDoc will likely need to be edited to reflect the actual changes after generation. -- `overwrite`: Whether to overwrite any existing PrDoc. - -### Sync - -Run sync and commit back results to PR. - -Posible combinations based on the `benchmark` dropdown. - -- `chain` - - Requires one of the following: - - `rococo` - - `westend` -- `sync-type` - - Requires one of the following: - - `warp` - - `full` - - `fast` - - `fast-unsafe` - -You can use the following [`gh cli`](https://cli.github.com/) inside the repo: - -```bash -gh workflow run command-sync.yml -f pr=1000 -f chain=rococo -f sync-type=full -``` - -## How to modify an action - -If you want to modify an action and test it, you can do by simply pushing your changes and then selecting your branch in the `Use worflow from` option. - -This will use a file from a specified branch. diff --git a/.github/scripts/cmd/_help.py b/.github/scripts/cmd/_help.py new file mode 100644 index 0000000000000..8ad49dad8461e --- /dev/null +++ b/.github/scripts/cmd/_help.py @@ -0,0 +1,26 @@ +import argparse + +""" + +Custom help action for argparse, it prints the help message for the main parser and all subparsers. + +""" + + +class _HelpAction(argparse._HelpAction): + def __call__(self, parser, namespace, values, option_string=None): + parser.print_help() + + # retrieve subparsers from parser + subparsers_actions = [ + action for action in parser._actions + if isinstance(action, argparse._SubParsersAction)] + # there will probably only be one subparser_action, + # but better save than sorry + for subparsers_action in subparsers_actions: + # get all subparsers and print help + for choice, subparser in subparsers_action.choices.items(): + print("\n### Command '{}'".format(choice)) + print(subparser.format_help()) + + parser.exit() diff --git a/.github/scripts/cmd/cmd.py b/.github/scripts/cmd/cmd.py new file mode 100755 index 0000000000000..63bd6a2795aac --- /dev/null +++ b/.github/scripts/cmd/cmd.py @@ -0,0 +1,196 @@ +#!/usr/bin/env python3 + +import os +import sys +import json +import argparse +import _help + +_HelpAction = _help._HelpAction + +f = open('.github/workflows/runtimes-matrix.json', 'r') +runtimesMatrix = json.load(f) + +runtimeNames = list(map(lambda x: x['name'], runtimesMatrix)) + +common_args = { + '--continue-on-fail': {"action": "store_true", "help": "Won't exit(1) on failed command and continue with next steps. "}, + '--quiet': {"action": "store_true", "help": "Won't print start/end/failed messages in PR"}, + '--clean': {"action": "store_true", "help": "Clean up the previous bot's & author's comments in PR"}, + '--image': {"help": "Override docker image '--image docker.io/paritytech/ci-unified:latest'"}, +} + +parser = argparse.ArgumentParser(prog="/cmd ", description='A command runner for polkadot-sdk repo', add_help=False) +parser.add_argument('--help', action=_HelpAction, help='help for help if you need some help') # help for help +for arg, config in common_args.items(): + parser.add_argument(arg, **config) + +subparsers = parser.add_subparsers(help='a command to run', dest='command') + +""" +BENCH +""" + +bench_example = '''**Examples**: + Runs all benchmarks + %(prog)s + + Runs benchmarks for pallet_balances and pallet_multisig for all runtimes which have these pallets. **--quiet** makes it to output nothing to PR but reactions + %(prog)s --pallet pallet_balances pallet_xcm_benchmarks::generic --quiet + + Runs bench for all pallets for westend runtime and continues even if some benchmarks fail + %(prog)s --runtime westend --continue-on-fail + + Does not output anything and cleans up the previous bot's & author command triggering comments in PR + %(prog)s --runtime westend rococo --pallet pallet_balances pallet_multisig --quiet --clean +''' + +parser_bench = subparsers.add_parser('bench', help='Runs benchmarks', epilog=bench_example, formatter_class=argparse.RawDescriptionHelpFormatter) + +for arg, config in common_args.items(): + parser_bench.add_argument(arg, **config) + +parser_bench.add_argument('--runtime', help='Runtime(s) space separated', choices=runtimeNames, nargs='*', default=runtimeNames) +parser_bench.add_argument('--pallet', help='Pallet(s) space separated', nargs='*', default=[]) + +""" +FMT +""" +parser_fmt = subparsers.add_parser('fmt', help='Formats code (cargo +nightly-VERSION fmt) and configs (taplo format)') +for arg, config in common_args.items(): + parser_fmt.add_argument(arg, **config) + +""" +Update UI +""" +parser_ui = subparsers.add_parser('update-ui', help='Updates UI tests') +for arg, config in common_args.items(): + parser_ui.add_argument(arg, **config) + + +args, unknown = parser.parse_known_args() + +print(f'args: {args}') + +if args.command == 'bench': + runtime_pallets_map = {} + failed_benchmarks = {} + successful_benchmarks = {} + + profile = "release" + + print(f'Provided runtimes: {args.runtime}') + # convert to mapped dict + runtimesMatrix = list(filter(lambda x: x['name'] in args.runtime, runtimesMatrix)) + runtimesMatrix = {x['name']: x for x in runtimesMatrix} + print(f'Filtered out runtimes: {runtimesMatrix}') + + # loop over remaining runtimes to collect available pallets + for runtime in runtimesMatrix.values(): + os.system(f"forklift cargo build -p {runtime['package']} --profile {profile} --features runtime-benchmarks") + print(f'-- listing pallets for benchmark for {runtime["name"]}') + wasm_file = f"target/{profile}/wbuild/{runtime['package']}/{runtime['package'].replace('-', '_')}.wasm" + output = os.popen( + f"frame-omni-bencher v1 benchmark pallet --no-csv-header --no-storage-info --no-min-squares --no-median-slopes --all --list --runtime={wasm_file}").read() + raw_pallets = output.strip().split('\n') + + all_pallets = set() + for pallet in raw_pallets: + if pallet: + all_pallets.add(pallet.split(',')[0].strip()) + + pallets = list(all_pallets) + print(f'Pallets in {runtime}: {pallets}') + runtime_pallets_map[runtime['name']] = pallets + + # filter out only the specified pallets from collected runtimes/pallets + if args.pallet: + print(f'Pallet: {args.pallet}') + new_pallets_map = {} + # keep only specified pallets if they exist in the runtime + for runtime in runtime_pallets_map: + if set(args.pallet).issubset(set(runtime_pallets_map[runtime])): + new_pallets_map[runtime] = args.pallet + + runtime_pallets_map = new_pallets_map + + print(f'Filtered out runtimes & pallets: {runtime_pallets_map}') + + if not runtime_pallets_map: + if args.pallet and not args.runtime: + print(f"No pallets {args.pallet} found in any runtime") + elif args.runtime and not args.pallet: + print(f"{args.runtime} runtime does not have any pallets") + elif args.runtime and args.pallet: + print(f"No pallets {args.pallet} found in {args.runtime}") + else: + print('No runtimes found') + sys.exit(1) + + header_path = os.path.abspath('./substrate/HEADER-APACHE2') + + for runtime in runtime_pallets_map: + for pallet in runtime_pallets_map[runtime]: + config = runtimesMatrix[runtime] + print(f'-- config: {config}') + if runtime == 'dev': + # to support sub-modules (https://github.com/paritytech/command-bot/issues/275) + search_manifest_path = f"cargo metadata --locked --format-version 1 --no-deps | jq -r '.packages[] | select(.name == \"{pallet.replace('_', '-')}\") | .manifest_path'" + print(f'-- running: {search_manifest_path}') + manifest_path = os.popen(search_manifest_path).read() + if not manifest_path: + print(f'-- pallet {pallet} not found in dev runtime') + exit(1) + package_dir = os.path.dirname(manifest_path) + print(f'-- package_dir: {package_dir}') + print(f'-- manifest_path: {manifest_path}') + output_path = os.path.join(package_dir, "src", "weights.rs") + else: + default_path = f"./{config['path']}/src/weights" + xcm_path = f"./{config['path']}/src/weights/xcm" + output_path = default_path if not pallet.startswith("pallet_xcm_benchmarks") else xcm_path + print(f'-- benchmarking {pallet} in {runtime} into {output_path}') + cmd = f"frame-omni-bencher v1 benchmark pallet --extrinsic=* --runtime=target/{profile}/wbuild/{config['package']}/{config['package'].replace('-', '_')}.wasm --pallet={pallet} --header={header_path} --output={output_path} --wasm-execution=compiled --steps=50 --repeat=20 --heap-pages=4096 --no-storage-info --no-min-squares --no-median-slopes" + print(f'-- Running: {cmd}') + status = os.system(cmd) + if status != 0 and not args.continue_on_fail: + print(f'Failed to benchmark {pallet} in {runtime}') + sys.exit(1) + + # Otherwise collect failed benchmarks and print them at the end + # push failed pallets to failed_benchmarks + if status != 0: + failed_benchmarks[f'{runtime}'] = failed_benchmarks.get(f'{runtime}', []) + [pallet] + else: + successful_benchmarks[f'{runtime}'] = successful_benchmarks.get(f'{runtime}', []) + [pallet] + + if failed_benchmarks: + print('❌ Failed benchmarks of runtimes/pallets:') + for runtime, pallets in failed_benchmarks.items(): + print(f'-- {runtime}: {pallets}') + + if successful_benchmarks: + print('βœ… Successful benchmarks of runtimes/pallets:') + for runtime, pallets in successful_benchmarks.items(): + print(f'-- {runtime}: {pallets}') + +elif args.command == 'fmt': + command = f"cargo +nightly fmt" + print(f'Formatting with `{command}`') + nightly_status = os.system(f'{command}') + taplo_status = os.system('taplo format --config .config/taplo.toml') + + if (nightly_status != 0 or taplo_status != 0) and not args.continue_on_fail: + print('❌ Failed to format code') + sys.exit(1) + +elif args.command == 'update-ui': + command = 'sh ./scripts/update-ui-tests.sh' + print(f'Updating ui with `{command}`') + status = os.system(f'{command}') + + if status != 0 and not args.continue_on_fail: + print('❌ Failed to format code') + sys.exit(1) + +print('πŸš€ Done') diff --git a/.github/workflows/cmd.yml b/.github/workflows/cmd.yml new file mode 100644 index 0000000000000..dac46cf435a67 --- /dev/null +++ b/.github/workflows/cmd.yml @@ -0,0 +1,411 @@ +name: Command + +on: + issue_comment: # listen for comments on issues + types: [ created ] + +permissions: # allow the action to comment on the PR + contents: write + issues: write + pull-requests: write + actions: read + +jobs: + is-org-member: + if: startsWith(github.event.comment.body, '/cmd') + runs-on: ubuntu-latest + outputs: + member: ${{ steps.is-member.outputs.result }} + steps: + - name: Generate token + id: generate_token + uses: tibdex/github-app-token@v2.1.0 + with: + app_id: ${{ secrets.CMD_BOT_APP_ID }} + private_key: ${{ secrets.CMD_BOT_APP_KEY }} + + - name: Check if user is a member of the organization + id: is-member + uses: actions/github-script@v7 + with: + github-token: ${{ steps.generate_token.outputs.token }} + result-encoding: string + script: | + const fs = require("fs"); + try { + const org = '${{ github.event.repository.owner.login }}'; + const username = '${{ github.event.comment.user.login }}'; + + const membership = await github.rest.orgs.checkMembershipForUser({ + org: org, + username: username + }); + + console.log(membership, membership.status, membership.status === 204); + + if (membership.status === 204) { + return 'true'; + } else { + console.log(membership); + fs.appendFileSync(process.env["GITHUB_STEP_SUMMARY"], `${membership.data && membership.data.message || 'Unknown error happened, please check logs'}`); + } + } catch (error) { + console.log(error) + } + + return 'false'; + + reject-non-members: + needs: is-org-member + if: ${{ startsWith(github.event.comment.body, '/cmd') && needs.is-org-member.outputs.member != 'true' }} + runs-on: ubuntu-latest + steps: + - name: Add reaction to rejected comment + uses: actions/github-script@v7 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + github.rest.reactions.createForIssueComment({ + comment_id: ${{ github.event.comment.id }}, + owner: context.repo.owner, + repo: context.repo.repo, + content: 'confused' + }) + + - name: Comment PR (Rejected) + uses: actions/github-script@v7 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: `Sorry, only members of the organization ${{ github.event.repository.owner.login }} members can run commands.` + }) + + acknowledge: + needs: is-org-member + if: ${{ startsWith(github.event.comment.body, '/cmd') && needs.is-org-member.outputs.member == 'true' }} + runs-on: ubuntu-latest + steps: + - name: Add reaction to triggered comment + uses: actions/github-script@v7 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + github.rest.reactions.createForIssueComment({ + comment_id: ${{ github.event.comment.id }}, + owner: context.repo.owner, + repo: context.repo.repo, + content: 'eyes' + }) + + clean: + needs: is-org-member + runs-on: ubuntu-latest + steps: + - name: Clean previous comments + if: ${{ startsWith(github.event.comment.body, '/cmd') && contains(github.event.comment.body, '--clean') && needs.is-org-member.outputs.member == 'true' }} + uses: actions/github-script@v7 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + github.rest.issues.listComments({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo + }).then(comments => { + for (let comment of comments.data) { + console.log(comment) + if ( + ${{ github.event.comment.id }} !== comment.id && + ( + ( + ( + comment.body.startsWith('Command') || + comment.body.startsWith('
Command') || + comment.body.startsWith('Sorry, only ') + ) && comment.user.type === 'Bot' + ) || + (comment.body.startsWith('/cmd') && comment.user.login === context.actor) + ) + ) { + github.rest.issues.deleteComment({ + comment_id: comment.id, + owner: context.repo.owner, + repo: context.repo.repo + }) + } + } + }) + help: + needs: [ clean, is-org-member ] + if: ${{ startsWith(github.event.comment.body, '/cmd') && contains(github.event.comment.body, '--help') && needs.is-org-member.outputs.member == 'true' }} + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Get command + uses: actions-ecosystem/action-regex-match@v2 + id: get-pr-comment + with: + text: ${{ github.event.comment.body }} + regex: '^(\/cmd )([-\/\s\w.=:]+)$' # see explanation in docs/contributor/commands-readme.md#examples + + - name: Save output of help + id: help + env: + CMD: ${{ steps.get-pr-comment.outputs.group2 }} # to avoid "" around the command + run: | + echo 'help<> $GITHUB_OUTPUT + python3 .github/scripts/cmd/cmd.py $CMD >> $GITHUB_OUTPUT + echo 'EOF' >> $GITHUB_OUTPUT + + - name: Comment PR (Help) + uses: actions/github-script@v7 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: `
Command help: + + \`\`\` + ${{ steps.help.outputs.help }} + \`\`\` + +
` + }) + + - name: Add confused reaction on failure + uses: actions/github-script@v7 + if: ${{ failure() }} + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + github.rest.reactions.createForIssueComment({ + comment_id: ${{ github.event.comment.id }}, + owner: context.repo.owner, + repo: context.repo.repo, + content: 'confused' + }) + + - name: Add πŸ‘ reaction on success + uses: actions/github-script@v7 + if: ${{ !failure() }} + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + github.rest.reactions.createForIssueComment({ + comment_id: ${{ github.event.comment.id }}, + owner: context.repo.owner, + repo: context.repo.repo, + content: '+1' + }) + + set-image: + needs: [ clean, is-org-member ] + if: ${{ startsWith(github.event.comment.body, '/cmd') && !contains(github.event.comment.body, '--help') && needs.is-org-member.outputs.member == 'true' }} + runs-on: ubuntu-latest + outputs: + IMAGE: ${{ steps.set-image.outputs.IMAGE }} + RUNNER: ${{ steps.set-image.outputs.RUNNER }} + steps: + - name: Checkout + uses: actions/checkout@v4 + + - id: set-image + run: | + BODY=$(echo "${{ github.event.comment.body }}" | xargs) + IMAGE_OVERRIDE=$(echo $BODY | grep -oe 'docker.io/paritytech/ci-unified:.*\s' | xargs) + + cat .github/env >> $GITHUB_OUTPUT + + if [ -n "$IMAGE_OVERRIDE" ]; then + echo "IMAGE=$IMAGE_OVERRIDE" >> $GITHUB_OUTPUT + fi + + if [[ $BODY == "/cmd bench"* ]]; then + echo "RUNNER=arc-runners-polkadot-sdk-benchmark" >> $GITHUB_OUTPUT + elif [[ $BODY == "/cmd update-ui"* ]]; then + echo "RUNNER=arc-runners-polkadot-sdk-beefy" >> $GITHUB_OUTPUT + else + echo "RUNNER=ubuntu-latest" >> $GITHUB_OUTPUT + fi + + cmd: + needs: [ set-image ] + env: + JOB_NAME: 'cmd' + runs-on: ${{ needs.set-image.outputs.RUNNER }} + container: + image: ${{ needs.set-image.outputs.IMAGE }} + steps: + - name: Get command + uses: actions-ecosystem/action-regex-match@v2 + id: get-pr-comment + with: + text: ${{ github.event.comment.body }} + regex: '^(\/cmd )([-\/\s\w.=:]+)$' # see explanation in docs/contributor/commands-readme.md#examples + + - name: Build workflow link + if: ${{ !contains(github.event.comment.body, '--quiet') }} + id: build-link + run: | + # Get exactly the CMD job link, filtering out the other jobs + jobLink=$(curl -s \ + -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \ + -H "Accept: application/vnd.github.v3+json" \ + https://api.github.com/repos/${{ github.repository }}/actions/runs/${{ github.run_id }}/jobs | jq '.jobs[] | select(.name | contains("${{ env.JOB_NAME }}")) | .html_url') + + runLink=$(curl -s \ + -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \ + -H "Accept: application/vnd.github.v3+json" \ + https://api.github.com/repos/${{ github.repository }}/actions/runs/${{ github.run_id }} | jq '.html_url') + + echo "job_url=${jobLink}" + echo "run_url=${runLink}" + echo "job_url=$jobLink" >> $GITHUB_OUTPUT + echo "run_url=$runLink" >> $GITHUB_OUTPUT + + + - name: Comment PR (Start) + if: ${{ !contains(github.event.comment.body, '--quiet') }} + uses: actions/github-script@v7 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + let job_url = ${{ steps.build-link.outputs.job_url }} + + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: `Command "${{ steps.get-pr-comment.outputs.group2 }}" has started πŸš€ [See logs here](${job_url})` + }) + + - name: Checkout + uses: actions/checkout@v4 + with: + ref: ${{ github.head_ref }} + + - name: Install dependencies for bench + if: startsWith(steps.get-pr-comment.outputs.group2, 'bench') + run: cargo install subweight frame-omni-bencher --locked + + - name: Run cmd + id: cmd + env: + CMD: ${{ steps.get-pr-comment.outputs.group2 }} # to avoid "" around the command + run: | + echo "Running command: '$CMD' on '${{ needs.set-image.outputs.RUNNER }}' runner, container: '${{ needs.set-image.outputs.IMAGE }}'" + echo "RUST_NIGHTLY_VERSION: $RUST_NIGHTLY_VERSION" + # Fixes "detected dubious ownership" error in the ci + git config --global --add safe.directory '*' + git remote -v + python3 .github/scripts/cmd/cmd.py $CMD + git status + git diff + + - name: Commit changes + run: | + if [ -n "$(git status --porcelain)" ]; then + git config --local user.email "action@github.com" + git config --local user.name "GitHub Action" + + git pull origin ${{ github.head_ref }} + git add . + git restore --staged Cargo.lock # ignore changes in Cargo.lock + git commit -m "Update from ${{ github.actor }} running command '${{ steps.get-pr-comment.outputs.group2 }}'" || true + git push origin ${{ github.head_ref }} + else + echo "Nothing to commit"; + fi + + - name: Run Subweight + id: subweight + if: startsWith(steps.get-pr-comment.outputs.group2, 'bench') + shell: bash + run: | + git fetch + result=$(subweight compare commits \ + --path-pattern "./**/weights/**/*.rs,./**/weights.rs" \ + --method asymptotic \ + --format markdown \ + --no-color \ + --change added changed \ + --ignore-errors \ + refs/remotes/origin/master ${{ github.ref }}) + + # Save the multiline result to the output + { + echo "result<> $GITHUB_OUTPUT + + - name: Comment PR (End) + if: ${{ !failure() && !contains(github.event.comment.body, '--quiet') }} + uses: actions/github-script@v7 + env: + SUBWEIGHT: '${{ steps.subweight.outputs.result }}' + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + let runUrl = ${{ steps.build-link.outputs.run_url }} + let subweight = process.env.SUBWEIGHT; + + let subweightCollapsed = subweight + ? `
\n\nSubweight results:\n\n${subweight}\n\n
` + : ''; + + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: `Command "${{ steps.get-pr-comment.outputs.group2 }}" has finished βœ… [See logs here](${runUrl})${subweightCollapsed}` + }) + + - name: Comment PR (Failure) + if: ${{ failure() && !contains(github.event.comment.body, '--quiet') }} + uses: actions/github-script@v7 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + let jobUrl = ${{ steps.build-link.outputs.job_url }} + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: `Command "${{ steps.get-pr-comment.outputs.group2 }}" has failed ❌! [See logs here](${jobUrl})` + }) + + - name: Add πŸ˜• reaction on failure + uses: actions/github-script@v7 + if: ${{ failure() }} + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + github.rest.reactions.createForIssueComment({ + comment_id: ${{ github.event.comment.id }}, + owner: context.repo.owner, + repo: context.repo.repo, + content: 'confused' + }) + + - name: Add πŸ‘ reaction on success + uses: actions/github-script@v7 + if: ${{ !failure() }} + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + github.rest.reactions.createForIssueComment({ + comment_id: ${{ github.event.comment.id }}, + owner: context.repo.owner, + repo: context.repo.repo, + content: '+1' + }) diff --git a/.github/workflows/command-bench-all.yml b/.github/workflows/command-bench-all.yml deleted file mode 100644 index 4128f86fb7c82..0000000000000 --- a/.github/workflows/command-bench-all.yml +++ /dev/null @@ -1,99 +0,0 @@ -name: Command Bench All - -on: - workflow_dispatch: - inputs: - pr: - description: Number of the Pull Request - required: true - benchmark: - description: Pallet benchmark - type: choice - required: true - options: - - pallet - - substrate - - polkadot - - cumulus - pallet: - description: Pallet - required: false - type: string - default: pallet_name - target_dir: - description: Target directory - type: choice - options: - - substrate - - polkadot - - cumulus - runtime: - description: Runtime - type: choice - options: - - rococo - - westend - - asset-hub-kusama - - asset-hub-polkadot - - asset-hub-rococo - - asset-hub-westend - - bridge-hub-kusama - - bridge-hub-polkadot - - bridge-hub-rococo - - bridge-hub-westend - - collectives-polkadot - - collectives-westend - - coretime-rococo - - coretime-westend - - contracts-rococo - - glutton-kusama - - glutton-westend - - people-rococo - - people-westend - -jobs: - set-image: - runs-on: ubuntu-latest - outputs: - IMAGE: ${{ steps.set_image.outputs.IMAGE }} - steps: - - name: Checkout - uses: actions/checkout@v4 - - id: set_image - run: cat .github/env >> $GITHUB_OUTPUT - cmd-bench-all: - needs: [set-image] - runs-on: arc-runners-polkadot-sdk-weights - container: - image: ${{ needs.set-image.outputs.IMAGE }} - permissions: - contents: write - pull-requests: write - steps: - - name: Download repo - uses: actions/checkout@v4 - - name: Install gh cli - id: gh - uses: ./.github/actions/set-up-gh - with: - pr-number: ${{ inputs.pr }} - GH_TOKEN: ${{ github.token }} - - name: Run bench all - run: | - "./scripts/bench-all.sh" "${{ inputs.benchmark }}" --runtime "${{ inputs.runtime }}" --pallet "${{ inputs.pallet }}" --target_dir "${{ inputs.target_dir }}" - - name: Report failure - if: ${{ failure() }} - run: gh pr comment ${{ inputs.pr }} --body "

Command failed ❌

Run by @${{ github.actor }} for ${{ github.workflow }} failed. See logs here." - env: - RUN: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_TOKEN: ${{ github.token }} - - run: git pull --rebase - - uses: stefanzweifel/git-auto-commit-action@v5 - with: - commit_message: cmd-action - ${{ github.workflow }} - branch: ${{ steps.gh.outputs.branch }} - - name: Report succeed - run: gh pr comment ${{ inputs.pr }} --body "

Action completed πŸŽ‰πŸŽ‰

Run by @${{ github.actor }} for ${{ github.workflow }} completed πŸŽ‰. See logs here." - env: - RUN: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_TOKEN: ${{ github.token }} diff --git a/.github/workflows/command-bench-overhead.yml b/.github/workflows/command-bench-overhead.yml deleted file mode 100644 index fec8d37bb9ef8..0000000000000 --- a/.github/workflows/command-bench-overhead.yml +++ /dev/null @@ -1,78 +0,0 @@ -name: Command Bench Overhead - -on: - workflow_dispatch: - inputs: - pr: - description: Number of the Pull Request - required: true - benchmark: - description: Pallet benchmark - type: choice - required: true - options: - - default - - substrate - - cumulus - runtime: - description: Runtime - type: choice - options: - - rococo - - westend - - asset-hub-rococo - - asset-hub-westend - target_dir: - description: Target directory - type: choice - options: - - polkadot - - substrate - - cumulus - -jobs: - set-image: - runs-on: ubuntu-latest - outputs: - IMAGE: ${{ steps.set_image.outputs.IMAGE }} - steps: - - name: Checkout - uses: actions/checkout@v4 - - id: set_image - run: cat .github/env >> $GITHUB_OUTPUT - cmd-bench-overhead: - needs: [set-image] - runs-on: arc-runners-polkadot-sdk-benchmark - container: - image: ${{ needs.set-image.outputs.IMAGE }} - permissions: - contents: write - pull-requests: write - steps: - - name: Download repo - uses: actions/checkout@v4 - - name: Install gh cli - id: gh - uses: ./.github/actions/set-up-gh - with: - pr-number: ${{ inputs.pr }} - GH_TOKEN: ${{ github.token }} - - name: Run bench overhead - run: | - "./scripts/bench.sh" "${{ inputs.benchmark }}" --subcommand "overhead" --runtime "${{ inputs.runtime }}" --target_dir "${{ inputs.target_dir }}" - - name: Report failure - if: ${{ failure() }} - run: gh pr comment ${{ inputs.pr }} --body "

Command failed ❌

Run by @${{ github.actor }} for ${{ github.workflow }} failed. See logs here." - env: - RUN: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_TOKEN: ${{ github.token }} - - run: git pull --rebase - - uses: stefanzweifel/git-auto-commit-action@v5 - with: - commit_message: cmd-action - ${{ github.workflow }} - branch: ${{ steps.gh.outputs.branch }} - - name: Report succeed - run: gh pr comment ${{ inputs.pr }} --body "

Action completed πŸŽ‰πŸŽ‰

Run by @${{ github.actor }} for ${{ github.workflow }} completed πŸŽ‰. See logs here." - env: - RUN: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_TOKEN: ${{ github.token }} diff --git a/.github/workflows/command-bench.yml b/.github/workflows/command-bench.yml deleted file mode 100644 index ac879f443755c..0000000000000 --- a/.github/workflows/command-bench.yml +++ /dev/null @@ -1,124 +0,0 @@ -name: Command Bench - -on: - workflow_dispatch: - inputs: - pr: - description: Number of the Pull Request - required: true - benchmark: - description: Pallet benchmark - type: choice - required: true - options: - - substrate-pallet - - polkadot-pallet - - cumulus-assets - - cumulus-collectives - - cumulus-coretime - - cumulus-bridge-hubs - - cumulus-contracts - - cumulus-glutton - - cumulus-starters - - cumulus-people - - cumulus-testing - subcommand: - description: Subcommand - type: choice - required: true - options: - - pallet - - xcm - runtime: - description: Runtime - type: choice - options: - - dev - - rococo - - westend - - asset-hub-westend - - asset-hub-rococo - - collectives-westend - - coretime-rococo - - coretime-westend - - bridge-hub-rococo - - bridge-hub-westend - - contracts-rococo - - glutton-westend - - glutton-westend-dev-1300 - - seedling - - shell - - people-westend - - people-rococo - - penpal - - rococo-parachain - pallet: - description: Pallet - type: string - default: pallet_name - target_dir: - description: Target directory - type: choice - options: - - substrate - - polkadot - - cumulus - runtime_dir: - description: Runtime directory - type: choice - options: - - people - - collectives - - coretime - - bridge-hubs - - contracts - - glutton - - starters - - testing - -jobs: - set-image: - runs-on: ubuntu-latest - outputs: - IMAGE: ${{ steps.set_image.outputs.IMAGE }} - steps: - - name: Checkout - uses: actions/checkout@v4 - - id: set_image - run: cat .github/env >> $GITHUB_OUTPUT - cmd-bench: - needs: [set-image] - runs-on: arc-runners-polkadot-sdk-benchmark - container: - image: ${{ needs.set-image.outputs.IMAGE }} - permissions: - contents: write - pull-requests: write - steps: - - name: Download repo - uses: actions/checkout@v4 - - name: Install gh cli - id: gh - uses: ./.github/actions/set-up-gh - with: - pr-number: ${{ inputs.pr }} - GH_TOKEN: ${{ github.token }} - - name: Run bench - run: | - "./scripts/bench.sh" "${{ inputs.benchmark }}" --runtime "${{ inputs.runtime }}" --pallet "${{ inputs.pallet }}" --target_dir "${{ inputs.target_dir }}" --subcommand "${{ inputs.subcommand }}" --runtime_dir "${{ inputs.runtime_dir }}" - - name: Report failure - if: ${{ failure() }} - run: gh pr comment ${{ inputs.pr }} --body "

Command failed ❌

Run by @${{ github.actor }} for ${{ github.workflow }} failed. See logs here." - env: - RUN: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_TOKEN: ${{ github.token }} - - run: git pull --rebase - - uses: stefanzweifel/git-auto-commit-action@v5 - with: - commit_message: cmd-action - ${{ github.workflow }} - branch: ${{ steps.gh.outputs.branch }} - - name: Report succeed - run: gh pr comment ${{ inputs.pr }} --body "

Action completed πŸŽ‰πŸŽ‰

Run by @${{ github.actor }} for ${{ github.workflow }} completed πŸŽ‰. See logs here." - env: - RUN: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_TOKEN: ${{ github.token }} diff --git a/.github/workflows/command-fmt.yml b/.github/workflows/command-fmt.yml deleted file mode 100644 index fc37a17ac549b..0000000000000 --- a/.github/workflows/command-fmt.yml +++ /dev/null @@ -1,65 +0,0 @@ -name: Command FMT - -on: - workflow_dispatch: - inputs: - pr: - description: Number of the Pull Request - required: true - -jobs: - set-image: - runs-on: ubuntu-latest - outputs: - IMAGE: ${{ steps.set_image.outputs.IMAGE }} - steps: - - name: Checkout - uses: actions/checkout@v4 - - id: set_image - run: cat .github/env >> $GITHUB_OUTPUT - cmd-fmt: - needs: [set-image] - runs-on: ubuntu-latest - timeout-minutes: 20 - container: - image: ${{ needs.set-image.outputs.IMAGE }} - permissions: - contents: write - pull-requests: write - steps: - - name: Download repo - uses: actions/checkout@v4 - - name: Install gh cli - id: gh - uses: ./.github/actions/set-up-gh - with: - pr-number: ${{ inputs.pr }} - GH_TOKEN: ${{ github.token }} - - name: Run FMT - run: | - cargo --version - rustc --version - cargo +nightly --version - rustc +nightly --version - - cargo +nightly fmt - - # format toml. - # since paritytech/ci-unified:bullseye-1.73.0-2023-11-01-v20231204 includes taplo-cli - taplo format --config .config/taplo.toml - - name: Report failure - if: ${{ failure() }} - run: gh pr comment ${{ inputs.pr }} --body "

Command failed ❌

Run by @${{ github.actor }} for ${{ github.workflow }} failed. See logs here." - env: - RUN: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_TOKEN: ${{ github.token }} - - run: git pull --rebase - - uses: stefanzweifel/git-auto-commit-action@v5 - with: - commit_message: cmd-action - ${{ github.workflow }} - branch: ${{ steps.gh.outputs.branch }} - - name: Report succeed - run: gh pr comment ${{ inputs.pr }} --body "

Action completed πŸŽ‰πŸŽ‰

Run by @${{ github.actor }} for ${{ github.workflow }} completed πŸŽ‰. See logs here." - env: - RUN: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_TOKEN: ${{ github.token }} diff --git a/.github/workflows/command-inform.yml b/.github/workflows/command-inform.yml deleted file mode 100644 index afdcf4c1b7b90..0000000000000 --- a/.github/workflows/command-inform.yml +++ /dev/null @@ -1,22 +0,0 @@ -name: Inform of new command action - -on: - issue_comment: - types: [created] - -jobs: - comment: - runs-on: ubuntu-latest - # Temporary disable the bot until the new command bot works properly - if: github.event.issue.pull_request && startsWith(github.event.comment.body, 'bot ') && false - steps: - - name: Inform that the new command exist - uses: actions/github-script@v7 - with: - script: | - github.rest.issues.createComment({ - issue_number: context.issue.number, - owner: context.repo.owner, - repo: context.repo.repo, - body: 'We are migrating the command bot to be a GitHub Action

Please, see the documentation on how to use it' - }) diff --git a/.github/workflows/command-prdoc.yml b/.github/workflows/command-prdoc.yml deleted file mode 100644 index 3a08b9a5fb286..0000000000000 --- a/.github/workflows/command-prdoc.yml +++ /dev/null @@ -1,90 +0,0 @@ -name: Command PrDoc - -on: - workflow_dispatch: - inputs: - pr: - type: number - description: Number of the Pull Request - required: true - bump: - type: choice - description: Default bump level for all crates - default: "TODO" - required: true - options: - - "TODO" - - "no change" - - "patch" - - "minor" - - "major" - audience: - type: choice - description: Audience of the PrDoc - default: "TODO" - required: true - options: - - "TODO" - - "Runtime Dev" - - "Runtime User" - - "Node Dev" - - "Node User" - overwrite: - type: choice - description: Overwrite existing PrDoc - default: "true" - required: true - options: - - "true" - - "false" - -concurrency: - group: command-prdoc - cancel-in-progress: true - -jobs: - set-image: - runs-on: ubuntu-latest - outputs: - IMAGE: ${{ steps.set_image.outputs.IMAGE }} - steps: - - name: Checkout - uses: actions/checkout@v4 - - id: set_image - run: cat .github/env >> $GITHUB_OUTPUT - cmd-prdoc: - needs: [set-image] - runs-on: ubuntu-latest - timeout-minutes: 20 - container: - image: ${{ needs.set-image.outputs.IMAGE }} - permissions: - contents: write - pull-requests: write - steps: - - name: Download repo - uses: actions/checkout@v4 - - name: Install gh cli - id: gh - uses: ./.github/actions/set-up-gh - with: - pr-number: ${{ inputs.pr }} - GH_TOKEN: ${{ github.token }} - - name: Generate PrDoc - run: | - python3 -m pip install -q cargo-workspace PyGithub whatthepatch pyyaml toml - - python3 .github/scripts/generate-prdoc.py --pr "${{ inputs.pr }}" --bump "${{ inputs.bump }}" --audience "${{ inputs.audience }}" --force "${{ inputs.overwrite }}" - - - name: Report failure - if: ${{ failure() }} - run: gh pr comment ${{ inputs.pr }} --body "

Command failed ❌

Run by @${{ github.actor }} for ${{ github.workflow }} failed. See logs here." - env: - RUN: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_TOKEN: ${{ github.token }} - - name: Push Commit - uses: stefanzweifel/git-auto-commit-action@v5 - with: - commit_message: Add PrDoc (auto generated) - branch: ${{ steps.gh.outputs.branch }} - file_pattern: 'prdoc/*.prdoc' diff --git a/.github/workflows/command-sync.yml b/.github/workflows/command-sync.yml deleted file mode 100644 index c610f4066a873..0000000000000 --- a/.github/workflows/command-sync.yml +++ /dev/null @@ -1,71 +0,0 @@ -name: Command Sync - -on: - workflow_dispatch: - inputs: - pr: - description: Number of the Pull Request - required: true - chain: - description: Chain - type: choice - required: true - options: - - westend - - rococo - sync-type: - description: Sync type - type: choice - required: true - options: - - warp - - full - - fast - - fast-unsafe - -jobs: - set-image: - runs-on: ubuntu-latest - outputs: - IMAGE: ${{ steps.set_image.outputs.IMAGE }} - steps: - - name: Checkout - uses: actions/checkout@v4 - - id: set_image - run: cat .github/env >> $GITHUB_OUTPUT - cmd-sync: - needs: [set-image] - runs-on: arc-runners-polkadot-sdk-warpsync - container: - image: ${{ needs.set-image.outputs.IMAGE }} - permissions: - contents: write - pull-requests: write - steps: - - name: Download repo - uses: actions/checkout@v4 - - name: Install gh cli - id: gh - uses: ./.github/actions/set-up-gh - with: - pr-number: ${{ inputs.pr }} - GH_TOKEN: ${{ github.token }} - - name: Run sync - run: | - "./scripts/sync.sh" --chain "${{ inputs.chain }}" --type "${{ inputs.sync-type }}" - - name: Report failure - if: ${{ failure() }} - run: gh pr comment ${{ inputs.pr }} --body "

Command failed ❌

Run by @${{ github.actor }} for ${{ github.workflow }} failed. See logs here." - env: - RUN: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_TOKEN: ${{ github.token }} - - run: git pull --rebase - - uses: stefanzweifel/git-auto-commit-action@v5 - with: - commit_message: cmd-action - ${{ github.workflow }} - branch: ${{ steps.gh.outputs.branch }} - - name: Report succeed - run: gh pr comment ${{ inputs.pr }} --body "

Action completed πŸŽ‰πŸŽ‰

Run by @${{ github.actor }} for ${{ github.workflow }} completed πŸŽ‰. See logs here." - env: - RUN: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_TOKEN: ${{ github.token }} diff --git a/.github/workflows/command-update-ui.yml b/.github/workflows/command-update-ui.yml deleted file mode 100644 index 860177adc8790..0000000000000 --- a/.github/workflows/command-update-ui.yml +++ /dev/null @@ -1,59 +0,0 @@ -name: Command Update UI - -on: - workflow_dispatch: - inputs: - pr: - description: Number of the Pull Request - required: true - rust-version: - description: Version of rust. Example 1.70 - required: false - -jobs: - set-image: - runs-on: ubuntu-latest - outputs: - IMAGE: ${{ steps.set_image.outputs.IMAGE }} - steps: - - name: Checkout - uses: actions/checkout@v4 - - id: set_image - run: cat .github/env >> $GITHUB_OUTPUT - cmd-update-ui: - needs: [set-image] - runs-on: arc-runners-polkadot-sdk-beefy - timeout-minutes: 90 - container: - image: ${{ needs.set-image.outputs.IMAGE }} - permissions: - contents: write - pull-requests: write - steps: - - name: Download repo - uses: actions/checkout@v4 - - name: Install gh cli - id: gh - uses: ./.github/actions/set-up-gh - with: - pr-number: ${{ inputs.pr }} - GH_TOKEN: ${{ github.token }} - - name: Run update-ui - run: | - "./scripts/update-ui-tests.sh" "${{ inputs.rust-version }}" - - name: Report failure - if: ${{ failure() }} - run: gh pr comment ${{ inputs.pr }} --body "

Command failed ❌

Run by @${{ github.actor }} for ${{ github.workflow }} failed. See logs here." - env: - RUN: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_TOKEN: ${{ github.token }} - - run: git pull --rebase - - uses: stefanzweifel/git-auto-commit-action@v5 - with: - commit_message: cmd-action - ${{ github.workflow }} - branch: ${{ steps.gh.outputs.branch }} - - name: Report succeed - run: gh pr comment ${{ inputs.pr }} --body "

Action completed πŸŽ‰πŸŽ‰

Run by @${{ github.actor }} for ${{ github.workflow }} completed πŸŽ‰. See logs here." - env: - RUN: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - GH_TOKEN: ${{ github.token }} diff --git a/.github/workflows/review-bot.yml b/.github/workflows/review-bot.yml index 80c96b0ef537f..3dd5b1114813d 100644 --- a/.github/workflows/review-bot.yml +++ b/.github/workflows/review-bot.yml @@ -15,7 +15,6 @@ on: jobs: review-approvals: runs-on: ubuntu-latest - environment: master steps: - name: Generate token id: app_token diff --git a/.github/workflows/runtimes-matrix.json b/.github/workflows/runtimes-matrix.json new file mode 100644 index 0000000000000..45a3acd3f166d --- /dev/null +++ b/.github/workflows/runtimes-matrix.json @@ -0,0 +1,98 @@ +[ + { + "name": "dev", + "package": "kitchensink-runtime", + "path": "substrate/frame", + "uri": null, + "is_relay": false + }, + { + "name": "westend", + "package": "westend-runtime", + "path": "polkadot/runtime/westend", + "uri": "wss://try-runtime-westend.polkadot.io:443", + "is_relay": true + }, + { + "name": "rococo", + "package": "rococo-runtime", + "path": "polkadot/runtime/rococo", + "uri": "wss://try-runtime-rococo.polkadot.io:443", + "is_relay": true + }, + { + "name": "asset-hub-westend", + "package": "asset-hub-westend-runtime", + "path": "cumulus/parachains/runtimes/assets/asset-hub-westend", + "uri": "wss://westend-asset-hub-rpc.polkadot.io:443", + "is_relay": false + }, + { + "name": "asset-hub-rococo", + "package": "asset-hub-rococo-runtime", + "path": "cumulus/parachains/runtimes/assets/asset-hub-rococo", + "uri": "wss://rococo-asset-hub-rpc.polkadot.io:443", + "is_relay": false + }, + { + "name": "bridge-hub-rococo", + "package": "bridge-hub-rococo-runtime", + "path": "cumulus/parachains/runtimes/bridges/bridge-hub-rococo", + "uri": "wss://rococo-bridge-hub-rpc.polkadot.io:443", + "is_relay": false + }, + { + "name": "bridge-hub-westend", + "package": "bridge-hub-rococo-runtime", + "path": "cumulus/parachains/runtimes/bridges/bridge-hub-westend", + "uri": "wss://westend-bridge-hub-rpc.polkadot.io:443", + "is_relay": false + }, + { + "name": "collectives-westend", + "package": "collectives-westend-runtime", + "path": "cumulus/parachains/runtimes/collectives/collectives-westend", + "uri": "wss://westend-collectives-rpc.polkadot.io:443" + }, + { + "name": "contracts-rococo", + "package": "contracts-rococo-runtime", + "path": "cumulus/parachains/runtimes/contracts/contracts-rococo", + "uri": "wss://rococo-contracts-rpc.polkadot.io:443", + "is_relay": false + }, + { + "name": "coretime-rococo", + "package": "coretime-rococo-runtime", + "path": "cumulus/parachains/runtimes/coretime/coretime-rococo", + "uri": "wss://rococo-coretime-rpc.polkadot.io:443", + "is_relay": false + }, + { + "name": "coretime-westend", + "package": "coretime-westend-runtime", + "path": "cumulus/parachains/runtimes/coretime/coretime-westend", + "uri": "wss://westend-coretime-rpc.polkadot.io:443", + "is_relay": false + }, + { + "name": "glutton-westend", + "package": "glutton-westend-runtime", + "path": "cumulus/parachains/runtimes/gluttons/glutton-westend", + "is_relay": false + }, + { + "name": "people-rococo", + "package": "people-rococo-runtime", + "path": "cumulus/parachains/runtimes/people/people-rococo", + "uri": "wss://rococo-people-rpc.polkadot.io:443", + "is_relay": false + }, + { + "name": "people-westend", + "package": "people-westend-runtime", + "path": "cumulus/parachains/runtimes/people/people-westend", + "uri": "wss://westend-people-rpc.polkadot.io:443", + "is_relay": false + } +] diff --git a/.gitignore b/.gitignore index e3e382af6195e..0263626d832df 100644 --- a/.gitignore +++ b/.gitignore @@ -23,6 +23,7 @@ **/node_modules **/target/ **/wip/*.stderr +**/__pycache__/ /.cargo/config /.envrc artifacts diff --git a/docs/contributor/CONTRIBUTING.md b/docs/contributor/CONTRIBUTING.md index 2e2d7a7fb4f7c..53f42b9ae4fb4 100644 --- a/docs/contributor/CONTRIBUTING.md +++ b/docs/contributor/CONTRIBUTING.md @@ -161,11 +161,11 @@ test output there is a script * `./scripts/update-ui-tests.sh` to update the tests for a current rust version locally * `./scripts/update-ui-tests.sh 1.70` # to update the tests for a specific rust version locally -Or if you have opened PR and you're member of `paritytech` - you can use command-bot to run the tests for you in CI: -* `bot update-ui` - will run the tests for the current rust version -* `bot update-ui latest --rust_version=1.70.0` - will run the tests for the specified rust version -* `bot update-ui latest -v CMD_IMAGE=paritytech/ci-unified:bullseye-1.70.0-2023-05-23 --rust_version=1.70.0` - will run -the tests for the specified rust version and specified image +Or if you have opened PR and you're member of `paritytech` - you can use [/cmd](./commands-readme.md) +to run the tests for you in CI: +* `/cmd update-ui` - will run the tests for the current rust version +* `/cmd update-ui --image docker.io/paritytech/ci-unified:bullseye-1.70.0-2023-05-23` - +will run the tests for the specified rust version and specified image ## Feature Propagation @@ -175,7 +175,7 @@ We use [zepter](https://github.com/ggwpez/zepter) to enforce features are propag If you're member of **paritytech** org - you can use command-bot to run various of common commands in CI: -Start with comment in PR: `bot help` to see the list of available commands. +Start with comment in PR: `/cmd --help` to see the list of available commands. ## Deprecating code diff --git a/docs/contributor/commands-readme.md b/docs/contributor/commands-readme.md new file mode 100644 index 0000000000000..2bb9bd7e7d582 --- /dev/null +++ b/docs/contributor/commands-readme.md @@ -0,0 +1,44 @@ +# Running Commands in PRs + +You can run commands in PRs by triggering it via comment. It will use the context of your PR and post the results back. +Note: it works only for members of the `paritytech` organization. + +## Usage + +`/cmd --help` to see all available commands and usage format + +`/cmd --help` to see the usage of a specific command + + +### Commands + +- `/cmd fmt` to format the code in the PR. It commits back with the formatted code (fmt) and configs (taplo). + +- `/cmd bench` to generate weights for a runtime. Read more about [Weight Generation](weight-generation.md) + +### Flags + +1.`--quiet` to suppress the output of the command in the comments. +By default, the Start and End/Failure of the command will be commented with the link to a pipeline. +If you want to avoid, use this flag. Go to +[Action Tab](https://github.com/paritytech/polkadot-sdk/actions/workflows/cmd.yml) to see the pipeline status. + +2.`--continue-on-fail` to continue running the command even if something inside a command +(like specific pallet weight generation) are failed. +Basically avoids interruption in the middle with `exit 1` +The pipeline logs will include what is failed (like which runtimes/pallets), then you can re-run them separately or not. + +3.`--clean` to clean up all yours and bot's comments in PR relevant to `/cmd` commands. If you run too many commands, +or they keep failing, and you're rerunning them again, it's handy to add this flag to keep a PR clean. + +### Adding new Commands +Feel free to add new commands to the workflow, however **_note_** that triggered workflows will use the actions +from `main` (default) branch, meaning they will take effect only after the PR with new changes/command is merged. +If you want to test the new command, it's better to test in your fork and local-to-fork PRs, where you control +the default branch. + +### Examples +The regex in cmd.yml is: `^(\/cmd )([-\/\s\w.=:]+)$` accepts only alphanumeric, space, "-", "/", "=", ":", "." chars. + +`/cmd bench --runtime bridge-hub-westend --pallet=pallet_name` +`/cmd update-ui --image=docker.io/paritytech/ci-unified:bullseye-1.77.0-2024-04-10-v202407161507 --clean` diff --git a/docs/contributor/weight-generation.md b/docs/contributor/weight-generation.md new file mode 100644 index 0000000000000..ebfdca59cae5d --- /dev/null +++ b/docs/contributor/weight-generation.md @@ -0,0 +1,69 @@ +# Weight Generation + +To generate weights for a runtime. +Weights generation is using self-hosted runner which is provided by Parity CI, the rest commands are using standard +GitHub runners on `ubuntu-latest` or `ubuntu-20.04`. +Self-hosted runner for benchmarks (arc-runners-Polkadot-sdk-benchmark) is configured to meet requirements of reference +hardware for running validators +https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#reference-hardware + +In a PR run the actions through comment: + +```sh +/cmd bench --help # outputs the actual usage documentation with examples and supported runtimes + +# or + +/cmd --help # to see all available commands +``` + +To regenerate all weights (however it will take long, +so don't do it unless you really need it), run the following command: +```sh +/cmd bench +``` + +To generate weights for all pallets in a particular runtime(s), run the following command: +```sh +/cmd bench --runtime kusama polkadot +``` + +For Substrate pallets (supports sub-modules too): +```sh +/cmd bench --runtime dev --pallet pallet_asset_conversion_ops +``` + +> **πŸ“ Note**: The action is not being run right-away, it will be queued and run in the next available runner. +So might be quick, but might also take up to 10 mins (That's in control of Github). +Once the action is run, you'll see reaction πŸ‘€ on original comment, and if you didn't pass `--quiet` - +it will also send a link to a pipeline when started, and link to whole workflow when finished. + +--- + +> **πŸ’‘Hint #1** : if you run all runtimes or all pallets, it might be that some pallet in the middle is failed +to generate weights, thus it stops (fails) the whole pipeline. +> If you want, you can make it to continue running, even if some pallets are failed, add `--continue-on-fail` +flag to the command. The report will include which runtimes/pallets have failed, then you can re-run +them separately after all is done. + +This way it runs all possible runtimes for the specified pallets, if it finds them in the runtime +```sh +/cmd bench --pallet pallet_balances pallet_xcm_benchmarks::generic pallet_xcm_benchmarks::fungible +``` + +If you want to run all specific pallet(s) for specific runtime(s), you can do it like this: +```sh +/cmd bench --runtime bridge-hub-polkadot --pallet pallet_xcm_benchmarks::generic pallet_xcm_benchmarks::fungible +``` + + +> **πŸ’‘Hint #2** : Sometimes when you run too many commands, or they keep failing and you're rerunning them again, +it's handy to add `--clean` flag to the command. This will clean up all yours and bot's comments in PR relevant to +/cmd commands. + +```sh +/cmd bench --runtime kusama polkadot --pallet=pallet_balances --clean --continue-on-fail +``` + +> **πŸ’‘Hint #3** : If you have questions or need help, feel free to tag @paritytech/opstooling (in github comments) +or ping in [matrix](https://matrix.to/#/#command-bot:parity.io) channel. diff --git a/scripts/bench-all.sh b/scripts/bench-all.sh deleted file mode 100755 index e5512e26bbad7..0000000000000 --- a/scripts/bench-all.sh +++ /dev/null @@ -1,16 +0,0 @@ -#!/usr/bin/env bash - -set -eu -o pipefail -shopt -s inherit_errexit -shopt -s globstar - -. "$(realpath "$(dirname "${BASH_SOURCE[0]}")/command-utils.sh")" - -get_arg optional --pallet "$@" -PALLET="${out:-""}" - -if [[ ! -z "$PALLET" ]]; then - . "$(dirname "${BASH_SOURCE[0]}")/lib/bench-all-pallet.sh" "$@" -else - . "$(dirname "${BASH_SOURCE[0]}")/bench.sh" --subcommand=all "$@" -fi diff --git a/scripts/bench.sh b/scripts/bench.sh deleted file mode 100755 index 2f4ef7ec6a141..0000000000000 --- a/scripts/bench.sh +++ /dev/null @@ -1,117 +0,0 @@ -#!/bin/bash -# Initially based on https://github.com/paritytech/bench-bot/blob/cd3b2943d911ae29e41fe6204788ef99c19412c3/bench.js - -# Most external variables used in this script, such as $GH_CONTRIBUTOR, are -# related to https://github.com/paritytech/try-runtime-bot - -# This script relies on $GITHUB_TOKEN which is probably a protected GitLab CI -# variable; if this assumption holds true, it is implied that this script should -# be ran only on protected pipelines - -set -eu -o pipefail -shopt -s inherit_errexit - -# realpath allows to reuse the current -BENCH_ROOT_DIR=$(realpath "$(dirname "${BASH_SOURCE[0]}")") - -. "$(realpath "$(dirname "${BASH_SOURCE[0]}")/command-utils.sh")" - -repository_name="$(basename "$PWD")" - -get_arg optional --target_dir "$@" -target_dir="${out:-""}" - -get_arg optional --noexit "$@" -noexit="${out:-""}" - -output_path="." - -profile="production" - -if [[ "$repository_name" == "polkadot-sdk" ]]; then - output_path="./$target_dir" -fi - -cargo_run_benchmarks="cargo run --quiet --profile=${profile}" - -echo "Repository: $repository_name" -echo "Target Dir: $target_dir" -echo "Output Path: $output_path" - -cargo_run() { - echo "Running $cargo_run_benchmarks" "${args[@]}" - - # if not patched with PATCH_something=123 then use --locked - if [[ -z "${BENCH_PATCHED:-}" ]]; then - cargo_run_benchmarks+=" --locked" - fi - - $cargo_run_benchmarks "${args[@]}" -} - - -main() { - - # Remove the "github" remote since the same repository might be reused by a - # GitLab runner, therefore the remote might already exist from a previous run - # in case it was not cleaned up properly for some reason - &>/dev/null git remote remove github || : - - tmp_dirs=() - cleanup() { - exit_code=$? - # Clean up the "github" remote at the end since it contains the - # $GITHUB_TOKEN secret, which is only available for protected pipelines on - # GitLab - &>/dev/null git remote remove github || : - rm -rf "${tmp_dirs[@]}" - echo "Done, exit: $exit_code" - exit $exit_code - } - - # avoid exit if --noexit is passed - if [ -z "$noexit" ]; then - trap cleanup EXIT - fi - - # set -x - - get_arg required --subcommand "$@" - local subcommand="${out:-""}" - - case "$subcommand" in - runtime|pallet|xcm) - echo 'Running bench_pallet' - . "$BENCH_ROOT_DIR/lib/bench-pallet.sh" "$@" - ;; - overhead) - echo 'Running bench_overhead' - . "$BENCH_ROOT_DIR/lib/bench-overhead.sh" "$@" - ;; - all) - echo "Running all-$target_dir" - . "$BENCH_ROOT_DIR/lib/bench-all-${target_dir}.sh" "$@" - ;; - *) - die "Invalid subcommand $subcommand to process_args" - ;; - esac - - # set +x - - # in case we used diener to patch some dependency during benchmark execution, - # revert the patches so that they're not included in the diff - git checkout --quiet HEAD Cargo.toml - - # Save the generated weights to GitLab artifacts in case commit+push fails - echo "Showing weights diff for command" - git diff -P | tee -a "${ARTIFACTS_DIR}/weights.patch" - echo "Wrote weights patch to \"${ARTIFACTS_DIR}/weights.patch\"" - - - # instead of using `cargo run --locked`, we allow the Cargo files to be updated - # but avoid committing them. It is so `cmd_runner_apply_patches` can work - git restore --staged Cargo.* -} - -main "$@" diff --git a/scripts/command-utils.sh b/scripts/command-utils.sh deleted file mode 100644 index 252e4c86480e6..0000000000000 --- a/scripts/command-utils.sh +++ /dev/null @@ -1,80 +0,0 @@ -#!/usr/bin/env bash - -if [ "${LOADED_UTILS_SH:-}" ]; then - return -else - export LOADED_UTILS_SH=true -fi - -export ARTIFACTS_DIR="$PWD/.git/.artifacts" - -die() { - if [ "${1:-}" ]; then - >&2 echo "$1" - fi - exit 1 -} - -get_arg() { - local arg_type="$1" - shift - - local is_required - case "$arg_type" in - required|required-many) - is_required=true - ;; - optional|optional-many) ;; - *) - die "Invalid is_required argument \"$2\" in get_arg" - ;; - esac - - local has_many_values - if [ "${arg_type: -6}" == "-many" ]; then - has_many_values=true - fi - - local option_arg="$1" - shift - - local args=("$@") - - unset out - out=() - - local get_next_arg - for arg in "${args[@]}"; do - if [ "${get_next_arg:-}" ]; then - out+=("$arg") - unset get_next_arg - if [ ! "${has_many_values:-}" ]; then - break - fi - # --foo=bar (get the value after '=') - elif [ "${arg:0:$(( ${#option_arg} + 1 ))}" == "$option_arg=" ]; then - out+=("${arg:$(( ${#option_arg} + 1 ))}") - if [ ! "${has_many_values:-}" ]; then - break - fi - # --foo bar (get the next argument) - elif [ "$arg" == "$option_arg" ]; then - get_next_arg=true - fi - done - - # arg list ended with --something but no argument was provided next - if [ "${get_next_arg:-}" ]; then - die "Expected argument after \"${args[-1]}"\" - fi - - if [ "${out[0]:-}" ]; then - if [ ! "${has_many_values:-}" ]; then - out="${out[0]}" - fi - elif [ "${is_required:-}" ]; then - die "Argument $option_arg is required, but was not found" - else - unset out - fi -} diff --git a/scripts/lib/bench-all-cumulus.sh b/scripts/lib/bench-all-cumulus.sh deleted file mode 100755 index f4c2a35c6b6b7..0000000000000 --- a/scripts/lib/bench-all-cumulus.sh +++ /dev/null @@ -1,139 +0,0 @@ -#!/usr/bin/env bash -# originally moved from https://github.com/paritytech/cumulus/blob/445f9277ab55b4d930ced4fbbb38d27c617c6658/scripts/benchmarks-ci.sh - -# default RUST_LOG is warn, but could be overridden -export RUST_LOG="${RUST_LOG:-error}" - -THIS_DIR=$(dirname "${BASH_SOURCE[0]}") -. "$THIS_DIR/../command-utils.sh" - -POLKADOT_PARACHAIN="./target/$profile/polkadot-parachain" - -run_cumulus_bench() { - local artifactsDir="$ARTIFACTS_DIR" - local category=$1 - local runtimeName=$2 - local paraId=${3:-} - - local benchmarkOutput="$output_path/parachains/runtimes/$category/$runtimeName/src/weights" - local benchmarkRuntimeChain - if [[ ! -z "$paraId" ]]; then - benchmarkRuntimeChain="${runtimeName}-dev-$paraId" - else - benchmarkRuntimeChain="$runtimeName-dev" - fi - - local benchmarkMetadataOutputDir="$artifactsDir/$runtimeName" - mkdir -p "$benchmarkMetadataOutputDir" - - # Load all pallet names in an array. - echo "[+] Listing pallets for runtime $runtimeName for chain: $benchmarkRuntimeChain ..." - local pallets=($( - $POLKADOT_PARACHAIN benchmark pallet --list --chain="${benchmarkRuntimeChain}" |\ - tail -n+2 |\ - cut -d',' -f1 |\ - sort |\ - uniq - )) - - if [ ${#pallets[@]} -ne 0 ]; then - echo "[+] Benchmarking ${#pallets[@]} pallets for runtime $runtimeName for chain: $benchmarkRuntimeChain, pallets:" - for pallet in "${pallets[@]}"; do - echo " [+] $pallet" - done - else - echo "$runtimeName pallet list not found in benchmarks-ci.sh" - exit 1 - fi - - for pallet in "${pallets[@]}"; do - # (by default) do not choose output_file, like `pallet_assets.rs` because it does not work for multiple instances - # `benchmark pallet` command will decide the output_file name if there are multiple instances - local output_file="" - local extra_args="" - # a little hack for pallet_xcm_benchmarks - we want to force custom implementation for XcmWeightInfo - if [[ "$pallet" == "pallet_xcm_benchmarks::generic" ]] || [[ "$pallet" == "pallet_xcm_benchmarks::fungible" ]]; then - output_file="xcm/${pallet//::/_}.rs" - extra_args="--template=$output_path/templates/xcm-bench-template.hbs" - fi - $POLKADOT_PARACHAIN benchmark pallet \ - $extra_args \ - --chain="${benchmarkRuntimeChain}" \ - --wasm-execution=compiled \ - --pallet="$pallet" \ - --no-storage-info \ - --no-median-slopes \ - --no-min-squares \ - --extrinsic='*' \ - --steps=50 \ - --repeat=20 \ - --json \ - --header="$output_path/file_header.txt" \ - --output="${benchmarkOutput}/${output_file}" >> "$benchmarkMetadataOutputDir/${pallet//::/_}_benchmark.json" - done -} - - -echo "[+] Compiling benchmarks..." -cargo build --profile $profile --locked --features=runtime-benchmarks -p polkadot-parachain-bin - -# Run benchmarks for all pallets of a given runtime if runtime argument provided -get_arg optional --runtime "$@" -runtime="${out:-""}" - -if [[ $runtime ]]; then - paraId="" - case "$runtime" in - asset-*) - category="assets" - ;; - collectives-*) - category="collectives" - ;; - coretime-*) - category="coretime" - ;; - bridge-*) - category="bridge-hubs" - ;; - contracts-*) - category="contracts" - ;; - people-*) - category="people" - ;; - glutton-*) - category="glutton" - paraId="1300" - ;; - *) - echo "Unknown runtime: $runtime" - exit 1 - ;; - esac - - run_cumulus_bench $category $runtime $paraId - -else # run all - # Assets - run_cumulus_bench assets asset-hub-rococo - run_cumulus_bench assets asset-hub-westend - - # Collectives - run_cumulus_bench collectives collectives-westend - - # Coretime - run_cumulus_bench coretime coretime-rococo - run_cumulus_bench coretime coretime-westend - - # People - run_cumulus_bench people people-rococo - run_cumulus_bench people people-westend - - # Bridge Hubs - run_cumulus_bench bridge-hubs bridge-hub-rococo - run_cumulus_bench bridge-hubs bridge-hub-westend - - # Glutton - run_cumulus_bench glutton glutton-westend 1300 -fi diff --git a/scripts/lib/bench-all-pallet.sh b/scripts/lib/bench-all-pallet.sh deleted file mode 100644 index e6908045ddbd7..0000000000000 --- a/scripts/lib/bench-all-pallet.sh +++ /dev/null @@ -1,96 +0,0 @@ -#!/usr/bin/env bash - -set -eu -o pipefail -shopt -s inherit_errexit -shopt -s globstar - -. "$(dirname "${BASH_SOURCE[0]}")/../command-utils.sh" - -get_arg required --pallet "$@" -PALLET="${out:-""}" - -REPO_NAME="$(basename "$PWD")" -BASE_COMMAND="$(dirname "${BASH_SOURCE[0]}")/../../bench/bench.sh --noexit=true --subcommand=pallet" - -WEIGHT_FILE_PATHS=( $(find . -type f -name "${PALLET}.rs" -path "**/weights/*" | sed 's|^\./||g') ) - -# convert pallet_ranked_collective to ranked-collective -CLEAN_PALLET=$(echo $PALLET | sed 's/pallet_//g' | sed 's/_/-/g') - -# add substrate pallet weights to a list -SUBSTRATE_PALLET_PATH=$(ls substrate/frame/$CLEAN_PALLET/src/weights.rs || :) -if [ ! -z "${SUBSTRATE_PALLET_PATH}" ]; then - WEIGHT_FILE_PATHS+=("$SUBSTRATE_PALLET_PATH") -fi - -# add trappist pallet weights to a list -TRAPPIST_PALLET_PATH=$(ls pallet/$CLEAN_PALLET/src/weights.rs || :) -if [ ! -z "${TRAPPIST_PALLET_PATH}" ]; then - WEIGHT_FILE_PATHS+=("$TRAPPIST_PALLET_PATH") -fi - -COMMANDS=() - -if [ "${#WEIGHT_FILE_PATHS[@]}" -eq 0 ]; then - echo "No weights files found for pallet: $PALLET" - exit 1 -else - echo "Found weights files for pallet: $PALLET" -fi - -for f in ${WEIGHT_FILE_PATHS[@]}; do - echo "- $f" - # f examples: - # cumulus/parachains/runtimes/assets/asset-hub-rococo/src/weights/pallet_balances.rs - # polkadot/runtime/rococo/src/weights/pallet_balances.rs - # runtime/trappist/src/weights/pallet_assets.rs - TARGET_DIR=$(echo $f | cut -d'/' -f 1) - - if [ "$REPO_NAME" == "polkadot-sdk" ]; then - case $TARGET_DIR in - cumulus) - TYPE=$(echo $f | cut -d'/' -f 2) - # Example: cumulus/parachains/runtimes/assets/asset-hub-rococo/src/weights/pallet_balances.rs - if [ "$TYPE" == "parachains" ]; then - RUNTIME=$(echo $f | cut -d'/' -f 5) - RUNTIME_DIR=$(echo $f | cut -d'/' -f 4) - COMMANDS+=("$BASE_COMMAND --runtime=$RUNTIME --runtime_dir=$RUNTIME_DIR --target_dir=$TARGET_DIR --pallet=$PALLET") - fi - ;; - polkadot) - # Example: polkadot/runtime/rococo/src/weights/pallet_balances.rs - RUNTIME=$(echo $f | cut -d'/' -f 3) - COMMANDS+=("$BASE_COMMAND --runtime=$RUNTIME --target_dir=$TARGET_DIR --pallet=$PALLET") - ;; - substrate) - # Example: substrate/frame/contracts/src/weights.rs - COMMANDS+=("$BASE_COMMAND --target_dir=$TARGET_DIR --runtime=dev --pallet=$PALLET") - ;; - *) - echo "Unknown dir: $TARGET_DIR" - exit 1 - ;; - esac - fi - - if [ "$REPO_NAME" == "trappist" ]; then - case $TARGET_DIR in - runtime) - TYPE=$(echo $f | cut -d'/' -f 2) - if [ "$TYPE" == "trappist" || "$TYPE" == "stout" ]; then - # Example: runtime/trappist/src/weights/pallet_assets.rs - COMMANDS+=("$BASE_COMMAND --target_dir=trappist --runtime=$TYPE --pallet=$PALLET") - fi - ;; - *) - echo "Unknown dir: $TARGET_DIR" - exit 1 - ;; - esac - fi -done - -for cmd in "${COMMANDS[@]}"; do - echo "Running command: $cmd" - . $cmd -done diff --git a/scripts/lib/bench-all-polkadot.sh b/scripts/lib/bench-all-polkadot.sh deleted file mode 100644 index ac52e00140e38..0000000000000 --- a/scripts/lib/bench-all-polkadot.sh +++ /dev/null @@ -1,88 +0,0 @@ -#!/bin/bash - -# Runs all benchmarks for all pallets, for a given runtime, provided by $1 -# Should be run on a reference machine to gain accurate benchmarks -# current reference machine: https://github.com/paritytech/polkadot/pull/6508/files -# original source: https://github.com/paritytech/polkadot/blob/b9842c4b52f6791fef6c11ecd020b22fe614f041/scripts/run_all_benches.sh - -get_arg required --runtime "$@" -runtime="${out:-""}" - -# default RUST_LOG is error, but could be overridden -export RUST_LOG="${RUST_LOG:-error}" - -echo "[+] Compiling benchmarks..." -cargo build --profile $profile --locked --features=runtime-benchmarks -p polkadot - -POLKADOT_BIN="./target/$profile/polkadot" - -# Update the block and extrinsic overhead weights. -echo "[+] Benchmarking block and extrinsic overheads..." -OUTPUT=$( - $POLKADOT_BIN benchmark overhead \ - --chain="${runtime}-dev" \ - --wasm-execution=compiled \ - --weight-path="$output_path/runtime/${runtime}/constants/src/weights/" \ - --warmup=10 \ - --repeat=100 \ - --header="$output_path/file_header.txt" -) -if [ $? -ne 0 ]; then - echo "$OUTPUT" >> "$ERR_FILE" - echo "[-] Failed to benchmark the block and extrinsic overheads. Error written to $ERR_FILE; continuing..." -fi - - -# Load all pallet names in an array. -PALLETS=($( - $POLKADOT_BIN benchmark pallet --list --chain="${runtime}-dev" |\ - tail -n+2 |\ - cut -d',' -f1 |\ - sort |\ - uniq -)) - -echo "[+] Benchmarking ${#PALLETS[@]} pallets for runtime $runtime" - -# Define the error file. -ERR_FILE="${ARTIFACTS_DIR}/benchmarking_errors.txt" -# Delete the error file before each run. -rm -f $ERR_FILE - -# Benchmark each pallet. -for PALLET in "${PALLETS[@]}"; do - echo "[+] Benchmarking $PALLET for $runtime"; - - output_file="" - if [[ $PALLET == *"::"* ]]; then - # translates e.g. "pallet_foo::bar" to "pallet_foo_bar" - output_file="${PALLET//::/_}.rs" - fi - - OUTPUT=$( - $POLKADOT_BIN benchmark pallet \ - --chain="${runtime}-dev" \ - --steps=50 \ - --repeat=20 \ - --no-storage-info \ - --no-median-slopes \ - --no-min-squares \ - --pallet="$PALLET" \ - --extrinsic="*" \ - --execution=wasm \ - --wasm-execution=compiled \ - --header="$output_path/file_header.txt" \ - --output="$output_path/runtime/${runtime}/src/weights/${output_file}" 2>&1 - ) - if [ $? -ne 0 ]; then - echo "$OUTPUT" >> "$ERR_FILE" - echo "[-] Failed to benchmark $PALLET. Error written to $ERR_FILE; continuing..." - fi -done - -# Check if the error file exists. -if [ -f "$ERR_FILE" ]; then - echo "[-] Some benchmarks failed. See: $ERR_FILE" -else - echo "[+] All benchmarks passed." -fi diff --git a/scripts/lib/bench-all-substrate.sh b/scripts/lib/bench-all-substrate.sh deleted file mode 100644 index eeb18cdd8bbb3..0000000000000 --- a/scripts/lib/bench-all-substrate.sh +++ /dev/null @@ -1,148 +0,0 @@ -#!/usr/bin/env bash - -# This file is part of Substrate. -# Copyright (C) 2022 Parity Technologies (UK) Ltd. -# SPDX-License-Identifier: Apache-2.0 -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# This script has three parts which all use the Substrate runtime: -# - Pallet benchmarking to update the pallet weights -# - Overhead benchmarking for the Extrinsic and Block weights -# - Machine benchmarking -# -# Should be run on a reference machine to gain accurate benchmarks -# current reference machine: https://github.com/paritytech/substrate/pull/5848 - -# Original source: https://github.com/paritytech/substrate/blob/ff9921a260a67e3a71f25c8b402cd5c7da787a96/scripts/run_all_benchmarks.sh -# Fail if any sub-command in a pipe fails, not just the last one. -set -o pipefail -# Fail on undeclared variables. -set -u -# Fail if any sub-command fails. -set -e -# Fail on traps. -# set -E - -# default RUST_LOG is warn, but could be overridden -export RUST_LOG="${RUST_LOG:-error}" - -echo "[+] Compiling Substrate benchmarks..." -cargo build --profile=$profile --locked --features=runtime-benchmarks -p staging-node-cli - -# The executable to use. -SUBSTRATE="./target/$profile/substrate-node" - -# Manually exclude some pallets. -EXCLUDED_PALLETS=( - # Helper pallets - "pallet_election_provider_support_benchmarking" - # Pallets without automatic benchmarking - "pallet_babe" - "pallet_grandpa" - "pallet_mmr" - "pallet_offences" - # Only used for testing, does not need real weights. - "frame_benchmarking_pallet_pov" - "pallet_example_tasks" - "pallet_example_basic" - "pallet_example_split" - "pallet_example_kitchensink" - "pallet_example_mbm" - "tasks_example" -) - -# Load all pallet names in an array. -ALL_PALLETS=($( - $SUBSTRATE benchmark pallet --list --chain=dev |\ - tail -n+2 |\ - cut -d',' -f1 |\ - sort |\ - uniq -)) - -# Define the error file. -ERR_FILE="${ARTIFACTS_DIR}/benchmarking_errors.txt" - -# Delete the error file before each run. -rm -f "$ERR_FILE" - -mkdir -p "$(dirname "$ERR_FILE")" - -# Update the block and extrinsic overhead weights. -echo "[+] Benchmarking block and extrinsic overheads..." -OUTPUT=$( - $SUBSTRATE benchmark overhead \ - --chain=dev \ - --wasm-execution=compiled \ - --weight-path="$output_path/frame/support/src/weights/" \ - --header="$output_path/HEADER-APACHE2" \ - --warmup=10 \ - --repeat=100 2>&1 -) -if [ $? -ne 0 ]; then - echo "$OUTPUT" >> "$ERR_FILE" - echo "[-] Failed to benchmark the block and extrinsic overheads. Error written to $ERR_FILE; continuing..." -fi - -echo "[+] Benchmarking ${#ALL_PALLETS[@]} Substrate pallets and excluding ${#EXCLUDED_PALLETS[@]}." - -echo "[+] Excluded pallets ${EXCLUDED_PALLETS[@]}" -echo "[+] ------ " -echo "[+] Whole list pallets ${ALL_PALLETS[@]}" - -# Benchmark each pallet. -for PALLET in "${ALL_PALLETS[@]}"; do - FOLDER="$(echo "${PALLET#*_}" | tr '_' '-')"; - WEIGHT_FILE="$output_path/frame/${FOLDER}/src/weights.rs" - - # Skip the pallet if it is in the excluded list. - - if [[ " ${EXCLUDED_PALLETS[@]} " =~ " ${PALLET} " ]]; then - echo "[+] Skipping $PALLET as it is in the excluded list." - continue - fi - - echo "[+] Benchmarking $PALLET with weight file $WEIGHT_FILE"; - - set +e # Disable exit on error for the benchmarking of the pallets - OUTPUT=$( - $SUBSTRATE benchmark pallet \ - --chain=dev \ - --steps=50 \ - --repeat=20 \ - --pallet="$PALLET" \ - --no-storage-info \ - --no-median-slopes \ - --no-min-squares \ - --extrinsic="*" \ - --wasm-execution=compiled \ - --heap-pages=4096 \ - --output="$WEIGHT_FILE" \ - --header="$output_path/HEADER-APACHE2" \ - --template="$output_path/.maintain/frame-weight-template.hbs" 2>&1 - ) - if [ $? -ne 0 ]; then - echo -e "$PALLET: $OUTPUT\n" >> "$ERR_FILE" - echo "[-] Failed to benchmark $PALLET. Error written to $ERR_FILE; continuing..." - fi - set -e # Re-enable exit on error -done - - -# Check if the error file exists. -if [ -s "$ERR_FILE" ]; then - echo "[-] Some benchmarks failed. See: $ERR_FILE" - exit 1 -else - echo "[+] All benchmarks passed." -fi diff --git a/scripts/lib/bench-overhead.sh b/scripts/lib/bench-overhead.sh deleted file mode 100644 index c4cca8b4c128c..0000000000000 --- a/scripts/lib/bench-overhead.sh +++ /dev/null @@ -1,66 +0,0 @@ -#!/bin/bash - -THIS_DIR=$(dirname "${BASH_SOURCE[0]}") -. "$THIS_DIR/../command-utils.sh" - -bench_overhead_common_args=( - -- - benchmark - overhead - --wasm-execution=compiled - --warmup=10 - --repeat=100 -) -bench_overhead() { - local args - case "$target_dir" in - substrate) - args=( - --bin=substrate - "${bench_overhead_common_args[@]}" - --header="$output_path/HEADER-APACHE2" - --weight-path="$output_path/frame/support/src/weights" - --chain="dev" - ) - ;; - polkadot) - get_arg required --runtime "$@" - local runtime="${out:-""}" - args=( - --bin=polkadot - "${bench_overhead_common_args[@]}" - --header="$output_path/file_header.txt" - --weight-path="$output_path/runtime/$runtime/constants/src/weights" - --chain="$runtime-dev" - ) - ;; - cumulus) - get_arg required --runtime "$@" - local runtime="${out:-""}" - args=( - -p=polkadot-parachain-bin - "${bench_overhead_common_args[@]}" - --header="$output_path/file_header.txt" - --weight-path="$output_path/parachains/runtimes/assets/$runtime/src/weights" - --chain="$runtime" - ) - ;; - trappist) - get_arg required --runtime "$@" - local runtime="${out:-""}" - args=( - "${bench_overhead_common_args[@]}" - --header="$output_path/templates/file_header.txt" - --weight-path="$output_path/runtime/$runtime/src/weights" - --chain="$runtime-dev" - ) - ;; - *) - die "Target Dir \"$target_dir\" is not supported in bench_overhead" - ;; - esac - - cargo_run "${args[@]}" -} - -bench_overhead "$@" diff --git a/scripts/lib/bench-pallet.sh b/scripts/lib/bench-pallet.sh deleted file mode 100644 index 15eac31e3a45c..0000000000000 --- a/scripts/lib/bench-pallet.sh +++ /dev/null @@ -1,178 +0,0 @@ -#!/bin/bash - -THIS_DIR=$(dirname "${BASH_SOURCE[0]}") -. "$THIS_DIR/../command-utils.sh" - -bench_pallet_common_args=( - -- - benchmark - pallet - --steps=50 - --repeat=20 - --extrinsic="*" - --wasm-execution=compiled - --heap-pages=4096 - --json-file="${ARTIFACTS_DIR}/bench.json" -) -bench_pallet() { - get_arg required --subcommand "$@" - local subcommand="${out:-""}" - - get_arg required --runtime "$@" - local runtime="${out:-""}" - - get_arg required --pallet "$@" - local pallet="${out:-""}" - - local args - case "$target_dir" in - substrate) - args=( - --features=runtime-benchmarks - --manifest-path="$output_path/bin/node/cli/Cargo.toml" - "${bench_pallet_common_args[@]}" - --pallet="$pallet" - --chain="$runtime" - ) - - case "$subcommand" in - pallet) - # Translates e.g. "pallet_foo::bar" to "pallet_foo_bar" - local output_dir="${pallet//::/_}" - - # Substrate benchmarks are output to the "frame" directory but they aren't - # named exactly after the $pallet argument. For example: - # - When $pallet == pallet_balances, the output folder is frame/balances - # - When $pallet == frame_benchmarking, the output folder is frame/benchmarking - # The common pattern we infer from those examples is that we should remove - # the prefix - if [[ "$output_dir" =~ ^[A-Za-z]*[^A-Za-z](.*)$ ]]; then - output_dir="${BASH_REMATCH[1]}" - fi - - # We also need to translate '_' to '-' due to the folders' naming - # conventions - output_dir="${output_dir//_/-}" - - args+=( - --header="$output_path/HEADER-APACHE2" - --output="$output_path/frame/$output_dir/src/weights.rs" - --template="$output_path/.maintain/frame-weight-template.hbs" - ) - ;; - *) - die "Subcommand $subcommand is not supported for $target_dir in bench_pallet" - ;; - esac - ;; - polkadot) - # For backward compatibility: replace "-dev" with "" - runtime=${runtime/-dev/} - - local weights_dir="$output_path/runtime/${runtime}/src/weights" - - args=( - --bin=polkadot - --features=runtime-benchmarks - "${bench_pallet_common_args[@]}" - --pallet="$pallet" - --chain="${runtime}-dev" - ) - - case "$subcommand" in - pallet) - args+=( - --header="$output_path/file_header.txt" - --output="${weights_dir}/" - ) - ;; - xcm) - args+=( - --header="$output_path/file_header.txt" - --template="$output_path/xcm/pallet-xcm-benchmarks/template.hbs" - --output="${weights_dir}/xcm/" - ) - ;; - *) - die "Subcommand $subcommand is not supported for $target_dir in bench_pallet" - ;; - esac - ;; - cumulus) - get_arg required --runtime_dir "$@" - local runtime_dir="${out:-""}" - local chain="$runtime" - - # to support specifying parachain id from runtime name (e.g. ["glutton-westend", "glutton-westend-dev-1300"]) - # If runtime ends with "-dev" or "-dev-\d+", leave as it is, otherwise concat "-dev" at the end of $chain - if [[ ! "$runtime" =~ -dev(-[0-9]+)?$ ]]; then - chain="${runtime}-dev" - fi - - # replace "-dev" or "-dev-\d+" with "" for runtime - runtime=$(echo "$runtime" | sed 's/-dev.*//g') - - args=( - -p=polkadot-parachain-bin - --features=runtime-benchmarks - "${bench_pallet_common_args[@]}" - --pallet="$pallet" - --chain="${chain}" - --header="$output_path/file_header.txt" - ) - - case "$subcommand" in - pallet) - args+=( - --output="$output_path/parachains/runtimes/$runtime_dir/$runtime/src/weights/" - ) - ;; - xcm) - mkdir -p "$output_path/parachains/runtimes/$runtime_dir/$runtime/src/weights/xcm" - args+=( - --template="$output_path/templates/xcm-bench-template.hbs" - --output="$output_path/parachains/runtimes/$runtime_dir/$runtime/src/weights/xcm/" - ) - ;; - *) - die "Subcommand $subcommand is not supported for $target_dir in bench_pallet" - ;; - esac - ;; - trappist) - local weights_dir="$output_path/runtime/$runtime/src/weights" - - args=( - --features=runtime-benchmarks - "${bench_pallet_common_args[@]}" - --pallet="$pallet" - --chain="${runtime}-dev" - --header="$output_path/templates/file_header.txt" - ) - - case "$subcommand" in - pallet) - args+=( - --output="${weights_dir}/" - ) - ;; - xcm) - args+=( - --template="$output_path/templates/xcm-bench-template.hbs" - --output="${weights_dir}/xcm/" - ) - ;; - *) - die "Subcommand $subcommand is not supported for $target_dir in bench_pallet" - ;; - esac - ;; - *) - die "Repository $target_dir is not supported in bench_pallet" - ;; - esac - - cargo_run "${args[@]}" -} - -bench_pallet "$@" diff --git a/scripts/sync.sh b/scripts/sync.sh deleted file mode 100755 index b5d8a52199371..0000000000000 --- a/scripts/sync.sh +++ /dev/null @@ -1,74 +0,0 @@ -#!/usr/bin/env bash - -set -eu -o pipefail - -. "$(realpath "$(dirname "${BASH_SOURCE[0]}")/command-utils.sh")" - - -# Function to check syncing status -check_syncing() { - # Send the system_health request and parse the isSyncing field - RESPONSE=$(curl -sSX POST http://127.0.0.1:9944 \ - --header 'Content-Type: application/json' \ - --data-raw '{"jsonrpc": "2.0", "method": "system_health", "params": [], "id": "1"}') - - # Check for errors in the curl command - if [ $? -ne 0 ]; then - echo "Error: Unable to send request to Polkadot node" - fi - - IS_SYNCING=$(echo $RESPONSE | jq -r '.result.isSyncing') - - # Check for errors in the jq command or missing field in the response - if [ $? -ne 0 ] || [ "$IS_SYNCING" == "null" ]; then - echo "Error: Unable to parse sync status from response" - fi - - # Return the isSyncing value - echo $IS_SYNCING -} - -main() { - get_arg required --chain "$@" - local chain="${out:-""}" - - get_arg required --type "$@" - local type="${out:-""}" - - export RUST_LOG="${RUST_LOG:-remote-ext=debug,runtime=trace}" - - cargo build --release - - cp "./target/release/polkadot" ./polkadot-bin - - # Start sync. - # "&" runs the process in the background - # "> /dev/tty" redirects the output of the process to the terminal - ./polkadot-bin --sync="$type" --chain="$chain" > "$ARTIFACTS_DIR/sync.log" 2>&1 & - - # Get the PID of process - POLKADOT_SYNC_PID=$! - - sleep 10 - - # Poll the node every 100 seconds until syncing is complete - while :; do - SYNC_STATUS="$(check_syncing)" - if [ "$SYNC_STATUS" == "true" ]; then - echo "Node is still syncing..." - sleep 100 - elif [ "$SYNC_STATUS" == "false" ]; then - echo "Node sync is complete!" - kill "$POLKADOT_SYNC_PID" # Stop the Polkadot node process once syncing is complete - exit 0 # Success - elif [[ "$SYNC_STATUS" = Error:* ]]; then - echo "$SYNC_STATUS" - exit 1 # Error - else - echo "Unknown error: $SYNC_STATUS" - exit 1 # Unknown error - fi - done -} - -main "$@" diff --git a/scripts/update-ui-tests.sh b/scripts/update-ui-tests.sh deleted file mode 100755 index d363e51e40414..0000000000000 --- a/scripts/update-ui-tests.sh +++ /dev/null @@ -1,41 +0,0 @@ -#!/usr/bin/env bash -# Script for updating the UI tests for a new rust stable version. -# Exit on error -set -e - -# by default current rust stable will be used -RUSTUP_RUN="" -# check if we have a parameter -# ./scripts/update-ui-tests.sh 1.70 -if [ ! -z "$1" ]; then - echo "RUST_VERSION: $1" - # This will run all UI tests with the rust stable 1.70. - # The script requires that rustup is installed. - RUST_VERSION=$1 - RUSTUP_RUN="rustup run $RUST_VERSION" - - - echo "installing rustup $RUST_VERSION" - if ! command -v rustup &> /dev/null - then - echo "rustup needs to be installed" - exit - fi - - rustup install $RUST_VERSION - rustup component add rust-src --toolchain $RUST_VERSION -fi - -# Ensure we run the ui tests -export RUN_UI_TESTS=1 -# We don't need any wasm files for ui tests -export SKIP_WASM_BUILD=1 -# Let trybuild overwrite the .stderr files -export TRYBUILD=overwrite - -# ./substrate -$RUSTUP_RUN cargo test --manifest-path substrate/primitives/runtime-interface/Cargo.toml ui -$RUSTUP_RUN cargo test -p sp-api-test ui -$RUSTUP_RUN cargo test -p frame-election-provider-solution-type ui -$RUSTUP_RUN cargo test -p frame-support-test --features=no-metadata-docs,try-runtime,experimental ui -$RUSTUP_RUN cargo test -p xcm-procedural ui