Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Introduce on_runtime_upgrade #5058

Merged

Conversation

shawntabrizi
Copy link
Member

@shawntabrizi shawntabrizi commented Feb 25, 2020

This is a basic implementation of the on_runtime_upgrade which triggers at the very beginning of block initialization, before system::initialize or module::on_initialize.

Introduces OnRuntimeUpgrade trait which exposes on_runtime_upgrade function which is triggered in the executor.

There is now an on_runtime_upgrade() function which can be defined in the decl_module macro.

We have also introduced a new storage item in FRAME System, RuntimeUpgraded which is a bool that expresses if a runtime upgrade occurred last block.

In the executive module, we read this storage item and execute the on_runtime_upgrade logic if needed.

Balances and Staking pallets have a new storage value StorageVersion which keeps track of a Releases enum, which will express the semantic version of the runtime storage, and should match 1:1 with the crate version.

Closes #4056

I have also removed traces of on_finalise and on_initialise as they have been deprecated since v1.0.

@shawntabrizi shawntabrizi changed the title Shawntabrizi runtime upgrade Introduce on_runtime_upgrade Feb 25, 2020
@shawntabrizi shawntabrizi added the A3-in_progress Pull request is in progress. No review needed at this stage. label Feb 25, 2020
@drewstone
Copy link
Contributor

Is this still planned?

@shawntabrizi
Copy link
Member Author

@drewstone Would like to, just need to prioritize 2.0 and avoid extra noise.

@shawntabrizi shawntabrizi reopened this Mar 3, 2020
shawntabrizi and others added 2 commits March 3, 2020 16:58
* adding unleash to ci (paritytech#5020)

* adding unleash to ci

* fixing formatting

* with a dot please

* alpha.3 now

* do not publish testing helpers

* remove old test-helpers cruft

* fix cargo.lock

* with alpha 4

* do not publish runtime-interface-test either

* disable more test crates from publishing

* switch to alpha.5

* replace tempdir with tempfile

* update lru

* switch to bytes 0.5

* release script fixes

* switch on and to latest alpha

* BUT THE SPACES

* Fix: CI failing for some CLI tests (paritytech#5043)

* Initial commit

Forked at: 41bb219
Parent branch: origin/master

* Increase killing grace period of CLI tests and display more info

* Use --dev everywhere possible

* Put pruning mode to its own params struct

* Add pruning params to export-blocks command

* Added missing file

* Removed not-dev mode in tests

* Add pruning mode to the revert command

* Decrease killing grace period again

* Move back unsafe_pruning to import_params

* Applied proposed changes

* aura: remove unused tx pool (paritytech#5046)

* aura: remove unused transaction pool parameter

* node-template: remove transaction pool from aura

* aura: fix tests

* Extend rust nightly detection in `wasm-builder` (paritytech#5021)

Instead of just testing `cargo` and `rustup run nightly`, we now test
the `CARGO` env variable and also scan non default nightlies. The user
is also now able to select the toolchain with `WASM_BUILD_TOOLCHAIN`.

* Add steps setting to benchmarking CLI (paritytech#5033)

* Add steps setting to CLI, use max value to hit worst case.

* Bump impl_version.

* Apply review suggestion.

* Remove indices from node-template (paritytech#5025)

* Remove indices from node-template

* Use identity lookup instead

* Bump impl

* clean cargo.toml

* Fix documentation for "BlockBuilder::push_trusted" (paritytech#5051)

* fix doc

* rephrase

* do not check unleash on every PR, only master and tags (paritytech#5054)

* do not check unleash on every PR, only master and tags

* move scripts folder

* add signed-tag check to CI

* remove publish-to-crates-io dependencies

Co-authored-by: s3krit <pugh@s3kr.it>

* prepare version to alpha.1 (paritytech#5055)

bump version to -alpha.1

* Sync: validate block responses for required data (paritytech#5052)

* Less verbose state-db logging

* Validate block responses for block bodies

* Update client/network/src/protocol.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Added validation test

* Disconnect on missing header as well

* Typo

Co-Authored-By: André Silva <andre.beat@gmail.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: André Silva <andre.beat@gmail.com>

* Make these chainspecs fields private (paritytech#5031)

* Fix dockerfile (paritytech#5059)

* Adds documentation for `wipe` and `commit` (paritytech#5053)

* Adds documentation for `wipe` and `commit`

This adds documentation to `wipe` and `commit` of `Externalities`.
Besides that it removes the default implementation that would just panic
and requires that all implementers of the trait implement the functions.

* Update primitives/externalities/src/lib.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Fix the issue with `trybuild`'s `ui` tests (paritytech#4992)

* repro ui bug

* fix the tests

* test with the new image

* test without CARGO_HOME

* test without fixes

* test again

* fix trybuild old versions

* bump CArgo.lock

* fix trybuild newest versions

* bump Cargo.lock

* trying on the latest image

* bump Cargo.lock

* run with the old image

* ci will be green on the image from 2020-02-19 [skip ci]

* bump Cargo.lock

* Activate publishing of draft releases... (paritytech#5062)

* Activate publishing of draft releases...

... And fix the message sending (missing parameter).

* publish_draft_release.sh now checks latest...

... release on github rather than just a tag

* Fix/div by zero (paritytech#5041)

* Handle gas_price being zero separately

* Bump spec_version

* Add a unit & integration tests for gas price = 0

* set missing metadata fields, prepping alpha.2 (paritytech#5067)

* setting first batch of descriptions

* fix what I just broke

* next batch

* and pallets, too

* last batch

* set cargo.lock

* keep'em dev-deps

* bump version to alpha.2

* Fix revalidation not revalidating multiple times (paritytech#5065)

* removes use of sc_client::Client from sc_finality_grandpa (paritytech#5030)

* removes use of sc_client::Client from sc_finality_grandpa

* code formatting

* code formatting

* removes use of sc_client::Client from sc_finality_grandpa

* Remove deprecated host functions (paritytech#5038)

Sadly we need to keep one function `ext_blake2_256`. This function is
manually defined in `sp-core`.

* removes use of sc_client::Client from sc_basic_authorship (paritytech#5050)

* removes use of sc-client from sc-basic-authorship

* refactor use of ProposerFactory

* correct dep path

* pallet-transaction-payment clean up (paritytech#5070)

* Formatting clean up

* Introduce separate setters for the fees.

* *: Rename prometheus-exporter crate to substrate-prometheus-end… (paritytech#5076)

This patch renames the crate for the following two reasons:

1. The prometheus-exporter crate introduces native in-process Prometheus
style instrumentation to the Substrate project. Within the Prometheus
ecosystem the term "exporter" is used for external processes exposing
metrics for e.g. the Linux Kernel. In-process exposition would be
described via the term "endpoint".

2. "prometheus-exporter" is generic and ignores the fact that it is only
usable within the context of Substrate. In addition the name
"prometheus-exporter" is already taken on crates.io.

* rename `browser-utils` to `substrate-browser-utils` (paritytech#5079)

* prepping for Alpha.3 (paritytech#5080)

* Bump to alpha.3

* update gitlab-ci

* Propagate DispatchError for benchmarks. (paritytech#5075)

* Propagate DispatchError for benchmarks.

* Apply review suggestions.

* Use RuntimeString.

* fix expect

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Add options to overwrite range bounds in benchmark command. (paritytech#5072)

* Add --mins --maxs to benchmark command.

* Apply review suggestions.

* Update yamux to version 0.4.4. (paritytech#5086)

* Remove more instances of futures01 (paritytech#4633)

* Start removing last few instances of futures01

* Use to_poll on wasm

* Revert "Use to_poll on wasm"

This reverts commit 1c61728.

* Fix fg test

* Upgrade network test futures

* Update offchain hyper version

* Update service test

* bump tokio to 0.2.10

* Removed some unneeded tokios

* fixes

* fix run_until_all_full

* Make service test debuggable

* Update client/offchain/src/api/http.rs

Co-Authored-By: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>

* Add service_test to test-int output

* nitpicking

* Finally fix test

* Give up and revert client/serviec/test

* Revert gitlab ci too

Co-authored-by: Demi Obenour <demi@parity.io>

* Make export blocks default to json on stdout (paritytech#5090)

* Make export blocks default to json on stdout

* Multiline instead of single line to stay under 100 cols

* Change --json flag to --binary, defaulting to json

* Offence reporting returns a result (paritytech#5082)

* Offence reporting returns a result

* Bump spec_version

* Use unwrap instead of assertions

* Fix more review grumbles

* Update to libp2p 0.16.2 (paritytech#5088)

* Remove request ID from the new protocol (paritytech#5049)

* Make sure we remove a peer on disconnect in gossip (paritytech#5104)

* Make sure we remove peers on disconnect in gossip state machine

* Clear up the code

* Add a comment

* Expose `state-db` memory info (paritytech#5110)

This exposes memory statistics from the state-db.

* Change extrinsic_count to extrinsic_index in pallet-utility (paritytech#5044)

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>

* Add more metrics to prometheus (paritytech#5034)

* Add a few things

* Add finality_grandpa_round

* fix fg tests

* Nitpicks

* Nitpicks

* Fix name of prometheus crate

* Update to SCALE 1.2.0 (paritytech#5113)

This updates `parity-scale-codec` to `1.2.0`, which includes multiple
performance improvements and a fix that bounds the capacity of a vector
at decoding.

* Lazy payouts (paritytech#4474)

* TODOs

* Remove superfluous:

* partial implementation

* full implementation

* fix preferences

* update comments

* upgrade test WIP

* fix more tests

* fix cutoff

* fix saturation

* comment

* upgrade mock

* upgrade test

* WIP migration

* WIP migration

* remove slot stake stuff

* fix merge

* migration of ledger

* remove equalize from test

* add test

* fix

* update doc

* fix compilation

* improve test readibility

* improve doc

* fix most todo

* fix migration and test

* remove println

* WIP

* add test and spec

* weight

* update doc

* safer end_era

* fix exposure of conversion

* Revert "safer end_era"

This reverts commit 72ff737.

* fix useless put

* exposure clipped

* doc

* fix payout with clipped

* fix node runtime

* add doc

* pluggable and generalized staking module

* remove print

* update doc

* refactor

* improve documentation and implementation

* fix test

* Fix test

* fix test

* fix test

* fix remove lowest stake from exposure, not biggest.

* nomination index arguments in nominator_payout

* add test

* try to fix offence

* apply slashed and bond eras until active era

* doc

* update spec version

* add test upgrade from previous test environment

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* nominators upgrade has been cleaned

* dynamic history depth implementation

* make current_era - history_depth included

* Change equality check to start era to less than or equal

* Use era specific validator prefs

* Add print statement and comment about start era if <

* fix next_reward overflow

* make more check for bad era claim for zero cost

* small refactor

* code refactor + fix use of deprecated storage

* fix wasm build

* add comment

* Fix tests

* remove outdated comment

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* gather active era information into one storage

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* impl on_runtime_upgrade

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Marcio Diaz <marcio@parity.io>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
Co-authored-by: s3krit <pugh@s3kr.it>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
Co-authored-by: Eric <e.lim0322@gmail.com>
Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Demi Obenour <demi@parity.io>
Co-authored-by: pscott <30843220+pscott@users.noreply.github.com>
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Copy link
Contributor

@gui1117 gui1117 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I very like the idea about having something to execute prior to on_initialize to avoid having conflict with other module.

I think saying that module must not call others in this function is a good contrain so all modules can update themself with no issue and not care about its public API.

(Maybe modules could opt-in for some on_runtime_upgrade API which is callable by other module during the on_runtime_upgrade function, by exposing a pub mod on_runtime_upgrade_module_api {..})

This doesn't solve any runtime upgrade logic that a runtime will need, it is intended to be used only for storage upgrade self contained inside a module IMHO.

In case a runtime wants to switch a module to another one or do big upgrade not self-contained, I think the logic should be written in node/runtime/src/lib.rs with an additional function. This function could call modules specific migration function for example:

fn runtime_upgrade() {
Balance::upgrade_from_other_currency_module(OldBalance);
// or
Balance::upgrade_path_for_kusama_1000();
}

And pallet_balance could expose this function under some cargo features,

frame/executive/src/lib.rs Outdated Show resolved Hide resolved
@shawntabrizi shawntabrizi added A0-please_review Pull request needs code review. B1-apinoteworthy and removed A3-in_progress Pull request is in progress. No review needed at this stage. labels Mar 3, 2020
@shawntabrizi shawntabrizi added this to the 2.0 milestone Mar 3, 2020
@shawntabrizi
Copy link
Member Author

Should be ready to review with the feedback given

Copy link
Contributor

@gui1117 gui1117 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

frame/staking/src/lib.rs Outdated Show resolved Hide resolved
shawntabrizi and others added 2 commits March 5, 2020 13:09
Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
@shawntabrizi shawntabrizi added A8-looksgood and removed A0-please_review Pull request needs code review. labels Mar 5, 2020
@shawntabrizi
Copy link
Member Author

@gnunicorn gucci gang

@gavofyork gavofyork merged commit eb41816 into paritytech:master Mar 5, 2020
@shawntabrizi shawntabrizi deleted the shawntabrizi-runtime-upgrade branch March 5, 2020 16:00
General-Beck pushed a commit to General-Beck/substrate that referenced this pull request Mar 6, 2020
* Initial idea of `on_runtime_upgrade`

* Runtime storage for module version

* Gui shawntabrizi runtime upgrade (paritytech#5118)

* adding unleash to ci (paritytech#5020)

* adding unleash to ci

* fixing formatting

* with a dot please

* alpha.3 now

* do not publish testing helpers

* remove old test-helpers cruft

* fix cargo.lock

* with alpha 4

* do not publish runtime-interface-test either

* disable more test crates from publishing

* switch to alpha.5

* replace tempdir with tempfile

* update lru

* switch to bytes 0.5

* release script fixes

* switch on and to latest alpha

* BUT THE SPACES

* Fix: CI failing for some CLI tests (paritytech#5043)

* Initial commit

Forked at: 41bb219
Parent branch: origin/master

* Increase killing grace period of CLI tests and display more info

* Use --dev everywhere possible

* Put pruning mode to its own params struct

* Add pruning params to export-blocks command

* Added missing file

* Removed not-dev mode in tests

* Add pruning mode to the revert command

* Decrease killing grace period again

* Move back unsafe_pruning to import_params

* Applied proposed changes

* aura: remove unused tx pool (paritytech#5046)

* aura: remove unused transaction pool parameter

* node-template: remove transaction pool from aura

* aura: fix tests

* Extend rust nightly detection in `wasm-builder` (paritytech#5021)

Instead of just testing `cargo` and `rustup run nightly`, we now test
the `CARGO` env variable and also scan non default nightlies. The user
is also now able to select the toolchain with `WASM_BUILD_TOOLCHAIN`.

* Add steps setting to benchmarking CLI (paritytech#5033)

* Add steps setting to CLI, use max value to hit worst case.

* Bump impl_version.

* Apply review suggestion.

* Remove indices from node-template (paritytech#5025)

* Remove indices from node-template

* Use identity lookup instead

* Bump impl

* clean cargo.toml

* Fix documentation for "BlockBuilder::push_trusted" (paritytech#5051)

* fix doc

* rephrase

* do not check unleash on every PR, only master and tags (paritytech#5054)

* do not check unleash on every PR, only master and tags

* move scripts folder

* add signed-tag check to CI

* remove publish-to-crates-io dependencies

Co-authored-by: s3krit <pugh@s3kr.it>

* prepare version to alpha.1 (paritytech#5055)

bump version to -alpha.1

* Sync: validate block responses for required data (paritytech#5052)

* Less verbose state-db logging

* Validate block responses for block bodies

* Update client/network/src/protocol.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Added validation test

* Disconnect on missing header as well

* Typo

Co-Authored-By: André Silva <andre.beat@gmail.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: André Silva <andre.beat@gmail.com>

* Make these chainspecs fields private (paritytech#5031)

* Fix dockerfile (paritytech#5059)

* Adds documentation for `wipe` and `commit` (paritytech#5053)

* Adds documentation for `wipe` and `commit`

This adds documentation to `wipe` and `commit` of `Externalities`.
Besides that it removes the default implementation that would just panic
and requires that all implementers of the trait implement the functions.

* Update primitives/externalities/src/lib.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Fix the issue with `trybuild`'s `ui` tests (paritytech#4992)

* repro ui bug

* fix the tests

* test with the new image

* test without CARGO_HOME

* test without fixes

* test again

* fix trybuild old versions

* bump CArgo.lock

* fix trybuild newest versions

* bump Cargo.lock

* trying on the latest image

* bump Cargo.lock

* run with the old image

* ci will be green on the image from 2020-02-19 [skip ci]

* bump Cargo.lock

* Activate publishing of draft releases... (paritytech#5062)

* Activate publishing of draft releases...

... And fix the message sending (missing parameter).

* publish_draft_release.sh now checks latest...

... release on github rather than just a tag

* Fix/div by zero (paritytech#5041)

* Handle gas_price being zero separately

* Bump spec_version

* Add a unit & integration tests for gas price = 0

* set missing metadata fields, prepping alpha.2 (paritytech#5067)

* setting first batch of descriptions

* fix what I just broke

* next batch

* and pallets, too

* last batch

* set cargo.lock

* keep'em dev-deps

* bump version to alpha.2

* Fix revalidation not revalidating multiple times (paritytech#5065)

* removes use of sc_client::Client from sc_finality_grandpa (paritytech#5030)

* removes use of sc_client::Client from sc_finality_grandpa

* code formatting

* code formatting

* removes use of sc_client::Client from sc_finality_grandpa

* Remove deprecated host functions (paritytech#5038)

Sadly we need to keep one function `ext_blake2_256`. This function is
manually defined in `sp-core`.

* removes use of sc_client::Client from sc_basic_authorship (paritytech#5050)

* removes use of sc-client from sc-basic-authorship

* refactor use of ProposerFactory

* correct dep path

* pallet-transaction-payment clean up (paritytech#5070)

* Formatting clean up

* Introduce separate setters for the fees.

* *: Rename prometheus-exporter crate to substrate-prometheus-end… (paritytech#5076)

This patch renames the crate for the following two reasons:

1. The prometheus-exporter crate introduces native in-process Prometheus
style instrumentation to the Substrate project. Within the Prometheus
ecosystem the term "exporter" is used for external processes exposing
metrics for e.g. the Linux Kernel. In-process exposition would be
described via the term "endpoint".

2. "prometheus-exporter" is generic and ignores the fact that it is only
usable within the context of Substrate. In addition the name
"prometheus-exporter" is already taken on crates.io.

* rename `browser-utils` to `substrate-browser-utils` (paritytech#5079)

* prepping for Alpha.3 (paritytech#5080)

* Bump to alpha.3

* update gitlab-ci

* Propagate DispatchError for benchmarks. (paritytech#5075)

* Propagate DispatchError for benchmarks.

* Apply review suggestions.

* Use RuntimeString.

* fix expect

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Add options to overwrite range bounds in benchmark command. (paritytech#5072)

* Add --mins --maxs to benchmark command.

* Apply review suggestions.

* Update yamux to version 0.4.4. (paritytech#5086)

* Remove more instances of futures01 (paritytech#4633)

* Start removing last few instances of futures01

* Use to_poll on wasm

* Revert "Use to_poll on wasm"

This reverts commit 1c61728.

* Fix fg test

* Upgrade network test futures

* Update offchain hyper version

* Update service test

* bump tokio to 0.2.10

* Removed some unneeded tokios

* fixes

* fix run_until_all_full

* Make service test debuggable

* Update client/offchain/src/api/http.rs

Co-Authored-By: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>

* Add service_test to test-int output

* nitpicking

* Finally fix test

* Give up and revert client/serviec/test

* Revert gitlab ci too

Co-authored-by: Demi Obenour <demi@parity.io>

* Make export blocks default to json on stdout (paritytech#5090)

* Make export blocks default to json on stdout

* Multiline instead of single line to stay under 100 cols

* Change --json flag to --binary, defaulting to json

* Offence reporting returns a result (paritytech#5082)

* Offence reporting returns a result

* Bump spec_version

* Use unwrap instead of assertions

* Fix more review grumbles

* Update to libp2p 0.16.2 (paritytech#5088)

* Remove request ID from the new protocol (paritytech#5049)

* Make sure we remove a peer on disconnect in gossip (paritytech#5104)

* Make sure we remove peers on disconnect in gossip state machine

* Clear up the code

* Add a comment

* Expose `state-db` memory info (paritytech#5110)

This exposes memory statistics from the state-db.

* Change extrinsic_count to extrinsic_index in pallet-utility (paritytech#5044)

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>

* Add more metrics to prometheus (paritytech#5034)

* Add a few things

* Add finality_grandpa_round

* fix fg tests

* Nitpicks

* Nitpicks

* Fix name of prometheus crate

* Update to SCALE 1.2.0 (paritytech#5113)

This updates `parity-scale-codec` to `1.2.0`, which includes multiple
performance improvements and a fix that bounds the capacity of a vector
at decoding.

* Lazy payouts (paritytech#4474)

* TODOs

* Remove superfluous:

* partial implementation

* full implementation

* fix preferences

* update comments

* upgrade test WIP

* fix more tests

* fix cutoff

* fix saturation

* comment

* upgrade mock

* upgrade test

* WIP migration

* WIP migration

* remove slot stake stuff

* fix merge

* migration of ledger

* remove equalize from test

* add test

* fix

* update doc

* fix compilation

* improve test readibility

* improve doc

* fix most todo

* fix migration and test

* remove println

* WIP

* add test and spec

* weight

* update doc

* safer end_era

* fix exposure of conversion

* Revert "safer end_era"

This reverts commit 72ff737.

* fix useless put

* exposure clipped

* doc

* fix payout with clipped

* fix node runtime

* add doc

* pluggable and generalized staking module

* remove print

* update doc

* refactor

* improve documentation and implementation

* fix test

* Fix test

* fix test

* fix test

* fix remove lowest stake from exposure, not biggest.

* nomination index arguments in nominator_payout

* add test

* try to fix offence

* apply slashed and bond eras until active era

* doc

* update spec version

* add test upgrade from previous test environment

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* nominators upgrade has been cleaned

* dynamic history depth implementation

* make current_era - history_depth included

* Change equality check to start era to less than or equal

* Use era specific validator prefs

* Add print statement and comment about start era if <

* fix next_reward overflow

* make more check for bad era claim for zero cost

* small refactor

* code refactor + fix use of deprecated storage

* fix wasm build

* add comment

* Fix tests

* remove outdated comment

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* gather active era information into one storage

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* impl on_runtime_upgrade

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Marcio Diaz <marcio@parity.io>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
Co-authored-by: s3krit <pugh@s3kr.it>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
Co-authored-by: Eric <e.lim0322@gmail.com>
Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Demi Obenour <demi@parity.io>
Co-authored-by: pscott <30843220+pscott@users.noreply.github.com>
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>

* make compile

* Add some tests

* docs

* Remove "useless" code

* Fix merge and use n + 1 block number

* Fix tests

* unfix ui tests

* Update on_initialize.stderr

* fix test

* Fix test

* Bump spec

* Remove `on_finalise` and `on_initialise`

* Use bool for tracking runtime upgraded

* typo

* Support runtime upgrade with `set_storage`

* Refactor migration code location

* add trailing newlines

* Remove old `IsUpgraded` flag

* Update state root

* Exhaustive match statement

* Apply suggestions from code review

Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Marcio Diaz <marcio@parity.io>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
Co-authored-by: s3krit <pugh@s3kr.it>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
Co-authored-by: Eric <e.lim0322@gmail.com>
Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Demi Obenour <demi@parity.io>
Co-authored-by: pscott <30843220+pscott@users.noreply.github.com>
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
General-Beck pushed a commit to General-Beck/substrate that referenced this pull request Mar 17, 2020
* Initial idea of `on_runtime_upgrade`

* Runtime storage for module version

* Gui shawntabrizi runtime upgrade (paritytech#5118)

* adding unleash to ci (paritytech#5020)

* adding unleash to ci

* fixing formatting

* with a dot please

* alpha.3 now

* do not publish testing helpers

* remove old test-helpers cruft

* fix cargo.lock

* with alpha 4

* do not publish runtime-interface-test either

* disable more test crates from publishing

* switch to alpha.5

* replace tempdir with tempfile

* update lru

* switch to bytes 0.5

* release script fixes

* switch on and to latest alpha

* BUT THE SPACES

* Fix: CI failing for some CLI tests (paritytech#5043)

* Initial commit

Forked at: 41bb219
Parent branch: origin/master

* Increase killing grace period of CLI tests and display more info

* Use --dev everywhere possible

* Put pruning mode to its own params struct

* Add pruning params to export-blocks command

* Added missing file

* Removed not-dev mode in tests

* Add pruning mode to the revert command

* Decrease killing grace period again

* Move back unsafe_pruning to import_params

* Applied proposed changes

* aura: remove unused tx pool (paritytech#5046)

* aura: remove unused transaction pool parameter

* node-template: remove transaction pool from aura

* aura: fix tests

* Extend rust nightly detection in `wasm-builder` (paritytech#5021)

Instead of just testing `cargo` and `rustup run nightly`, we now test
the `CARGO` env variable and also scan non default nightlies. The user
is also now able to select the toolchain with `WASM_BUILD_TOOLCHAIN`.

* Add steps setting to benchmarking CLI (paritytech#5033)

* Add steps setting to CLI, use max value to hit worst case.

* Bump impl_version.

* Apply review suggestion.

* Remove indices from node-template (paritytech#5025)

* Remove indices from node-template

* Use identity lookup instead

* Bump impl

* clean cargo.toml

* Fix documentation for "BlockBuilder::push_trusted" (paritytech#5051)

* fix doc

* rephrase

* do not check unleash on every PR, only master and tags (paritytech#5054)

* do not check unleash on every PR, only master and tags

* move scripts folder

* add signed-tag check to CI

* remove publish-to-crates-io dependencies

Co-authored-by: s3krit <pugh@s3kr.it>

* prepare version to alpha.1 (paritytech#5055)

bump version to -alpha.1

* Sync: validate block responses for required data (paritytech#5052)

* Less verbose state-db logging

* Validate block responses for block bodies

* Update client/network/src/protocol.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Added validation test

* Disconnect on missing header as well

* Typo

Co-Authored-By: André Silva <andre.beat@gmail.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: André Silva <andre.beat@gmail.com>

* Make these chainspecs fields private (paritytech#5031)

* Fix dockerfile (paritytech#5059)

* Adds documentation for `wipe` and `commit` (paritytech#5053)

* Adds documentation for `wipe` and `commit`

This adds documentation to `wipe` and `commit` of `Externalities`.
Besides that it removes the default implementation that would just panic
and requires that all implementers of the trait implement the functions.

* Update primitives/externalities/src/lib.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Fix the issue with `trybuild`'s `ui` tests (paritytech#4992)

* repro ui bug

* fix the tests

* test with the new image

* test without CARGO_HOME

* test without fixes

* test again

* fix trybuild old versions

* bump CArgo.lock

* fix trybuild newest versions

* bump Cargo.lock

* trying on the latest image

* bump Cargo.lock

* run with the old image

* ci will be green on the image from 2020-02-19 [skip ci]

* bump Cargo.lock

* Activate publishing of draft releases... (paritytech#5062)

* Activate publishing of draft releases...

... And fix the message sending (missing parameter).

* publish_draft_release.sh now checks latest...

... release on github rather than just a tag

* Fix/div by zero (paritytech#5041)

* Handle gas_price being zero separately

* Bump spec_version

* Add a unit & integration tests for gas price = 0

* set missing metadata fields, prepping alpha.2 (paritytech#5067)

* setting first batch of descriptions

* fix what I just broke

* next batch

* and pallets, too

* last batch

* set cargo.lock

* keep'em dev-deps

* bump version to alpha.2

* Fix revalidation not revalidating multiple times (paritytech#5065)

* removes use of sc_client::Client from sc_finality_grandpa (paritytech#5030)

* removes use of sc_client::Client from sc_finality_grandpa

* code formatting

* code formatting

* removes use of sc_client::Client from sc_finality_grandpa

* Remove deprecated host functions (paritytech#5038)

Sadly we need to keep one function `ext_blake2_256`. This function is
manually defined in `sp-core`.

* removes use of sc_client::Client from sc_basic_authorship (paritytech#5050)

* removes use of sc-client from sc-basic-authorship

* refactor use of ProposerFactory

* correct dep path

* pallet-transaction-payment clean up (paritytech#5070)

* Formatting clean up

* Introduce separate setters for the fees.

* *: Rename prometheus-exporter crate to substrate-prometheus-end… (paritytech#5076)

This patch renames the crate for the following two reasons:

1. The prometheus-exporter crate introduces native in-process Prometheus
style instrumentation to the Substrate project. Within the Prometheus
ecosystem the term "exporter" is used for external processes exposing
metrics for e.g. the Linux Kernel. In-process exposition would be
described via the term "endpoint".

2. "prometheus-exporter" is generic and ignores the fact that it is only
usable within the context of Substrate. In addition the name
"prometheus-exporter" is already taken on crates.io.

* rename `browser-utils` to `substrate-browser-utils` (paritytech#5079)

* prepping for Alpha.3 (paritytech#5080)

* Bump to alpha.3

* update gitlab-ci

* Propagate DispatchError for benchmarks. (paritytech#5075)

* Propagate DispatchError for benchmarks.

* Apply review suggestions.

* Use RuntimeString.

* fix expect

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Add options to overwrite range bounds in benchmark command. (paritytech#5072)

* Add --mins --maxs to benchmark command.

* Apply review suggestions.

* Update yamux to version 0.4.4. (paritytech#5086)

* Remove more instances of futures01 (paritytech#4633)

* Start removing last few instances of futures01

* Use to_poll on wasm

* Revert "Use to_poll on wasm"

This reverts commit 1c61728.

* Fix fg test

* Upgrade network test futures

* Update offchain hyper version

* Update service test

* bump tokio to 0.2.10

* Removed some unneeded tokios

* fixes

* fix run_until_all_full

* Make service test debuggable

* Update client/offchain/src/api/http.rs

Co-Authored-By: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>

* Add service_test to test-int output

* nitpicking

* Finally fix test

* Give up and revert client/serviec/test

* Revert gitlab ci too

Co-authored-by: Demi Obenour <demi@parity.io>

* Make export blocks default to json on stdout (paritytech#5090)

* Make export blocks default to json on stdout

* Multiline instead of single line to stay under 100 cols

* Change --json flag to --binary, defaulting to json

* Offence reporting returns a result (paritytech#5082)

* Offence reporting returns a result

* Bump spec_version

* Use unwrap instead of assertions

* Fix more review grumbles

* Update to libp2p 0.16.2 (paritytech#5088)

* Remove request ID from the new protocol (paritytech#5049)

* Make sure we remove a peer on disconnect in gossip (paritytech#5104)

* Make sure we remove peers on disconnect in gossip state machine

* Clear up the code

* Add a comment

* Expose `state-db` memory info (paritytech#5110)

This exposes memory statistics from the state-db.

* Change extrinsic_count to extrinsic_index in pallet-utility (paritytech#5044)

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>

* Add more metrics to prometheus (paritytech#5034)

* Add a few things

* Add finality_grandpa_round

* fix fg tests

* Nitpicks

* Nitpicks

* Fix name of prometheus crate

* Update to SCALE 1.2.0 (paritytech#5113)

This updates `parity-scale-codec` to `1.2.0`, which includes multiple
performance improvements and a fix that bounds the capacity of a vector
at decoding.

* Lazy payouts (paritytech#4474)

* TODOs

* Remove superfluous:

* partial implementation

* full implementation

* fix preferences

* update comments

* upgrade test WIP

* fix more tests

* fix cutoff

* fix saturation

* comment

* upgrade mock

* upgrade test

* WIP migration

* WIP migration

* remove slot stake stuff

* fix merge

* migration of ledger

* remove equalize from test

* add test

* fix

* update doc

* fix compilation

* improve test readibility

* improve doc

* fix most todo

* fix migration and test

* remove println

* WIP

* add test and spec

* weight

* update doc

* safer end_era

* fix exposure of conversion

* Revert "safer end_era"

This reverts commit 72ff737.

* fix useless put

* exposure clipped

* doc

* fix payout with clipped

* fix node runtime

* add doc

* pluggable and generalized staking module

* remove print

* update doc

* refactor

* improve documentation and implementation

* fix test

* Fix test

* fix test

* fix test

* fix remove lowest stake from exposure, not biggest.

* nomination index arguments in nominator_payout

* add test

* try to fix offence

* apply slashed and bond eras until active era

* doc

* update spec version

* add test upgrade from previous test environment

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* nominators upgrade has been cleaned

* dynamic history depth implementation

* make current_era - history_depth included

* Change equality check to start era to less than or equal

* Use era specific validator prefs

* Add print statement and comment about start era if <

* fix next_reward overflow

* make more check for bad era claim for zero cost

* small refactor

* code refactor + fix use of deprecated storage

* fix wasm build

* add comment

* Fix tests

* remove outdated comment

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* gather active era information into one storage

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* impl on_runtime_upgrade

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Marcio Diaz <marcio@parity.io>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
Co-authored-by: s3krit <pugh@s3kr.it>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
Co-authored-by: Eric <e.lim0322@gmail.com>
Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Demi Obenour <demi@parity.io>
Co-authored-by: pscott <30843220+pscott@users.noreply.github.com>
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>

* make compile

* Add some tests

* docs

* Remove "useless" code

* Fix merge and use n + 1 block number

* Fix tests

* unfix ui tests

* Update on_initialize.stderr

* fix test

* Fix test

* Bump spec

* Remove `on_finalise` and `on_initialise`

* Use bool for tracking runtime upgraded

* typo

* Support runtime upgrade with `set_storage`

* Refactor migration code location

* add trailing newlines

* Remove old `IsUpgraded` flag

* Update state root

* Exhaustive match statement

* Apply suggestions from code review

Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Marcio Diaz <marcio@parity.io>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
Co-authored-by: s3krit <pugh@s3kr.it>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
Co-authored-by: Eric <e.lim0322@gmail.com>
Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Demi Obenour <demi@parity.io>
Co-authored-by: pscott <30843220+pscott@users.noreply.github.com>
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Introduce on_runtime_upgrade similarly to on_initialize but executed before.
8 participants