Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add build options to the srtool build step #4956

Merged
merged 24 commits into from
Aug 26, 2024

Conversation

EgorPopelyaev
Copy link
Contributor

This PR adds possibility to set BUILD_OPTIONS to the "Srtool Build" step in the release pipeline while building runtimes.

Colses: https://github.com/paritytech/release-engineering/issues/213

@EgorPopelyaev EgorPopelyaev added the R0-silent Changes should not be mentioned in any release notes label Jul 5, 2024
@EgorPopelyaev EgorPopelyaev requested a review from a team July 5, 2024 15:11
@EgorPopelyaev EgorPopelyaev requested review from a team as code owners July 5, 2024 15:11
run: |
BUILD_OPTIONS=""
if [[ ${{ matrix.chain }} == 'asset-hub-rococo' || ${{ matrix.chain }} == 'asset-hub-westend' || ${{ matrix.chain }} == 'rococo' || ${{ matrix.chain }} == 'westend' ]]; then
BUILD_OPTIONS="${{ inputs.build_opts }}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conceptually it seems a bit odd that it ignores the build opts for some runtimes. Maybe you can add a METADATA_HASH_ENABLED flag or something?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add on-chain-release-build to every runtime and use this one here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the initial plan, the reason why I went this way was, that it includes this feature sp-api/disable-logging that turns off the logging, that is on the other hand needed for devops guys cc: @PierreBesson. After discussion with Pierre, that was kinda compromised solution🙈 to not build two types of runtimes (with on chain feature and with logging). If there is another to do it, I'll be happy to use it:). Otherwise, I can also add additional step to build those runtimes with logs for devops.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are also running logging nodes for main net, because we have logging disabled there. However, we can also just remove the "disable-logging" feature there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested the solution with two separate jobs, building runtimes with and without logging (having on-chain-release-build option). It looks fine. The only thing now is that people-rococo and poepele-westend seems not have this feature. Whom could I contact to ask to add this option? Is it Joe?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just copy the feature declaration from the other runtimes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And then we should remove the disable-logging everywhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basti, I've added the on-chain-release-build where it was missing, should the disable-logging deletion be done here as well or as a separate task? I'm not sure if it is worth to mention it in the rel notes or not.

@ggwpez
Copy link
Member

ggwpez commented Jul 5, 2024

I think we should generally re-evaluate if we still need srtool at all, since we should not in theory.
But since the current release process relies on it, its probably fine to do this later.

@EgorPopelyaev
Copy link
Contributor Author

I think we should generally re-evaluate if we still need srtool at all, since we should not in theory. But since the current release process relies on it, its probably fine to do this later.

Yep, I agree with that, we can do it step by step. After the switch to the new process, we can start to re-think what is needed for the current setup and what can be replaced

@paritytech-review-bot paritytech-review-bot bot requested a review from a team July 8, 2024 07:53
@bkchr
Copy link
Member

bkchr commented Jul 8, 2024

I think we should generally re-evaluate if we still need srtool at all, since we should not in theory. But since the current release process relies on it, its probably fine to do this later.

srtool only exist to have reproducible builds. There is no other reason for its existence. I'm not sure what the current state of rust and reproducible builds is. If we don't need it anymore, we can remove it.

uses: "./.github/workflows/release-srtool.yml"
with:
excluded_runtimes: "substrate-test bp cumulus-test kitchensink minimal-template parachain-template penpal polkadot-test seedling shell frame-try sp solochain-template"
build_opts: ""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't enable the on-chain-release-build feature. This means that metadata hash will not be generated at compile time. This is bad 🙈

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this one was done only for the case to have the logging enabled. In theory, this step should go away if the disable-logging feature will be deleted from the on-chain-release-build as we discussed above. If it is ok, I can delete it as part of this PR, and then we will have only the one build with the on-chain-release-build feature activated.

@EgorPopelyaev
Copy link
Contributor Author

@bkchr @ggwpez Basti, Oli, I've deleted the disable-logging from the on-chain-release-build feature as discussed above and adjusted the flow so, that we build the runtimes only once with on-chain-release-build activated. Could you please have a look if it is fine so and PR can be merged?

Copy link

Command failed ❌

Run by @EgorPopelyaev for Command PrDoc failed. See logs here.

@EgorPopelyaev
Copy link
Contributor Author

@bkchr Basti, could you please have a look at the deletion of the disable-logging features and if it is ok to merge so?

@bkchr bkchr added this pull request to the merge queue Aug 26, 2024
Merged via the queue into paritytech:master with commit 3cbefaf Aug 26, 2024
188 of 190 checks passed
ordian added a commit that referenced this pull request Aug 27, 2024
* master: (36 commits)
  Bump the ci_dependencies group across 1 directory with 2 updates (#5401)
  Remove deprecated calls in cumulus-parachain-system (#5439)
  Make the PR template a default for new PRs (#5462)
  Only log the propagating transactions when they are not empty (#5424)
  [CI] Fix SemVer check base commit (#5361)
  Sync status refactoring (#5450)
  Add build options to the srtool build step (#4956)
  `MaybeConsideration` extension trait for `Consideration` (#5384)
  Skip slot before creating inherent data providers during major sync (#5344)
  Add symlinks for code of conduct and contribution guidelines (#5447)
  pallet-collator-selection: correctly register weight in `new_session` (#5430)
  Derive `Clone` on `EncodableOpaqueLeaf` (#5442)
  Moving `Find FAIL-CI` check to GHA (#5377)
  Remove panic, as proof is invalid. (#5427)
  Reactive syncing metrics (#5410)
  [bridges] Prune messages from confirmation tx body, not from the on_idle (#5006)
  Change the chain to Rococo in the parachain template Zombienet config (#5279)
  Improve the appearance of crates on `crates.io` (#5243)
  Add initial version of `pallet_revive` (#5293)
  Update OpenZeppelin template documentation (#5398)
  ...
@EgorPopelyaev EgorPopelyaev deleted the ep-add-build-opts-to-release branch October 11, 2024 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
R0-silent Changes should not be mentioned in any release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants