Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Detect storage migration requirements in CI #233

Open
ggwpez opened this issue Jan 11, 2023 · 3 comments
Open

Detect storage migration requirements in CI #233

ggwpez opened this issue Jan 11, 2023 · 3 comments
Labels
I5-enhancement An additional feature request. T1-FRAME This PR/Issue is related to core FRAME, the framework.

Comments

@ggwpez
Copy link
Member

ggwpez commented Jan 11, 2023

Sometimes it is not obvious if a type is stored in storage and would need a storage migration.
We can automate this detection with Subwasm by applying it to the Kitchensink runtime, which has all (?) pallets deployed.
Subwasm can detect metadata changes to storage definitions. Such a change always necessitate a migration.
We can then add a label or somehow silence the check one a migration was added.

I opened this issue here so we can play around with it a bit in Substrate, and once its proven to work roll it out to Polkadot and Cumulus as well.
@ggwpez
Copy link
Member Author

ggwpez commented Mar 28, 2023

One recent example where we forgot a migration was in: paritytech/substrate#13128.
It migrated the BondedPools map, but not the RewardPools map which had to be hot-fixed in paritytech/substrate#13715.
So the check can be tested with that MR to detect both maps as "must migrate".

@kianenigma
Copy link
Contributor

Subwasm can detect metadata changes to storage definitions. Such a change always necessitate a migration.

Stumbled upon https://github.com/hack-ink/subalfred-check-runtime-action which can also do the same, and it is already packaged as a GH action.

@juangirini juangirini transferred this issue from paritytech/substrate Aug 24, 2023
@the-right-joyce the-right-joyce added I5-enhancement An additional feature request. T1-FRAME This PR/Issue is related to core FRAME, the framework. and removed J0-enhancement labels Aug 25, 2023
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 8, 2024
@HCastano @serban300
Fix Local Dockerfile Entrypoint (paritytech#233)
d6c50db 
* Correctly resolve project in Dockerfile entrypoint

* Improve instruction for running local Docker containers

* Rename bridge entrypoint script
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 8, 2024
@HCastano @serban300
Fix Local Dockerfile Entrypoint (paritytech#233)
b227293 
* Correctly resolve project in Dockerfile entrypoint

* Improve instruction for running local Docker containers

* Rename bridge entrypoint script
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
@HCastano @serban300
Fix Local Dockerfile Entrypoint (paritytech#233)
9371008 
* Correctly resolve project in Dockerfile entrypoint

* Improve instruction for running local Docker containers

* Rename bridge entrypoint script
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
@HCastano @serban300
Fix Local Dockerfile Entrypoint (paritytech#233)
f2394d7 
* Correctly resolve project in Dockerfile entrypoint

* Improve instruction for running local Docker containers

* Rename bridge entrypoint script
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
@HCastano @serban300
Fix Local Dockerfile Entrypoint (paritytech#233)
a1baf28 
* Correctly resolve project in Dockerfile entrypoint

* Improve instruction for running local Docker containers

* Rename bridge entrypoint script
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
@HCastano @serban300
Fix Local Dockerfile Entrypoint (paritytech#233)
35da04c 
* Correctly resolve project in Dockerfile entrypoint

* Improve instruction for running local Docker containers

* Rename bridge entrypoint script
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
@HCastano @serban300
Fix Local Dockerfile Entrypoint (paritytech#233)
1f61d4e 
* Correctly resolve project in Dockerfile entrypoint

* Improve instruction for running local Docker containers

* Rename bridge entrypoint script
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
@HCastano @serban300
Fix Local Dockerfile Entrypoint (paritytech#233)
133794c 
* Correctly resolve project in Dockerfile entrypoint

* Improve instruction for running local Docker containers

* Rename bridge entrypoint script
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 10, 2024
@HCastano @serban300
Fix Local Dockerfile Entrypoint (paritytech#233)
4fa8179 
* Correctly resolve project in Dockerfile entrypoint

* Improve instruction for running local Docker containers

* Rename bridge entrypoint script
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 10, 2024
@HCastano @serban300
Fix Local Dockerfile Entrypoint (paritytech#233)
6618a01 
* Correctly resolve project in Dockerfile entrypoint

* Improve instruction for running local Docker containers

* Rename bridge entrypoint script
jonathanudd pushed a commit to jonathanudd/polkadot-sdk that referenced this issue Apr 10, 2024
@HCastano @bkchr
Fix Local Dockerfile Entrypoint (paritytech#233)
70135cb 
* Correctly resolve project in Dockerfile entrypoint

* Improve instruction for running local Docker containers

* Rename bridge entrypoint script
github-merge-queue bot pushed a commit that referenced this issue Nov 5, 2024
@lexnv @dmitry-markin
litep2p: Update litep2p to v0.8.0 (#6353)
94389a9 
This PR updates litep2p to the latest release.

- `KademliaEvent::PutRecordSucess` is renamed to fix word typo
- `KademliaEvent::GetProvidersSuccess` and
`KademliaEvent::IncomingProvider` are needed for bootnodes on DHT work
and will be utilized later


### Added

- kad: Providers part 8: unit, e2e, and `libp2p` conformance tests
([#258](paritytech/litep2p#258))
- kad: Providers part 7: better types and public API, public addresses &
known providers ([#246](paritytech/litep2p#246))
- kad: Providers part 6: stop providing
([#245](paritytech/litep2p#245))
- kad: Providers part 5: `GET_PROVIDERS` query
([#236](paritytech/litep2p#236))
- kad: Providers part 4: refresh local providers
([#235](paritytech/litep2p#235))
- kad: Providers part 3: publish provider records (start providing)
([#234](paritytech/litep2p#234))

### Changed

- transport_service: Improve connection stability by downgrading
connections on substream inactivity
([#260](paritytech/litep2p#260))
- transport: Abort canceled dial attempts for TCP, WebSocket and Quic
([#255](paritytech/litep2p#255))
- kad/executor: Add timeout for writting frames
([#277](paritytech/litep2p#277))
- kad: Avoid cloning the `KademliaMessage` and use reference for
`RoutingTable::closest`
([#233](paritytech/litep2p#233))
- peer_state: Robust state machine transitions
([#251](paritytech/litep2p#251))
- address_store: Improve address tracking and add eviction algorithm
([#250](paritytech/litep2p#250))
- kad: Remove unused serde cfg
([#262](paritytech/litep2p#262))
- req-resp: Refactor to move functionality to dedicated methods
([#244](paritytech/litep2p#244))
- transport_service: Improve logs and move code from tokio::select macro
([#254](paritytech/litep2p#254))

### Fixed

- tcp/websocket/quic: Fix cancel memory leak
([#272](paritytech/litep2p#272))
- transport: Fix pending dials memory leak
([#271](paritytech/litep2p#271))
- ping: Fix memory leak of unremoved `pending_opens`
([#274](paritytech/litep2p#274))
- identify: Fix memory leak of unused `pending_opens`
([#273](paritytech/litep2p#273))
- kad: Fix not retrieving local records
([#221](paritytech/litep2p#221))

See release changelog for more details:
https://github.com/paritytech/litep2p/releases/tag/v0.8.0

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
lexnv added a commit that referenced this issue Nov 15, 2024
@lexnv @dmitry-markin
litep2p: Update litep2p to v0.8.0 (#6353)
1cac380 
This PR updates litep2p to the latest release.

- `KademliaEvent::PutRecordSucess` is renamed to fix word typo
- `KademliaEvent::GetProvidersSuccess` and
`KademliaEvent::IncomingProvider` are needed for bootnodes on DHT work
and will be utilized later

- kad: Providers part 8: unit, e2e, and `libp2p` conformance tests
([#258](paritytech/litep2p#258))
- kad: Providers part 7: better types and public API, public addresses &
known providers ([#246](paritytech/litep2p#246))
- kad: Providers part 6: stop providing
([#245](paritytech/litep2p#245))
- kad: Providers part 5: `GET_PROVIDERS` query
([#236](paritytech/litep2p#236))
- kad: Providers part 4: refresh local providers
([#235](paritytech/litep2p#235))
- kad: Providers part 3: publish provider records (start providing)
([#234](paritytech/litep2p#234))

- transport_service: Improve connection stability by downgrading
connections on substream inactivity
([#260](paritytech/litep2p#260))
- transport: Abort canceled dial attempts for TCP, WebSocket and Quic
([#255](paritytech/litep2p#255))
- kad/executor: Add timeout for writting frames
([#277](paritytech/litep2p#277))
- kad: Avoid cloning the `KademliaMessage` and use reference for
`RoutingTable::closest`
([#233](paritytech/litep2p#233))
- peer_state: Robust state machine transitions
([#251](paritytech/litep2p#251))
- address_store: Improve address tracking and add eviction algorithm
([#250](paritytech/litep2p#250))
- kad: Remove unused serde cfg
([#262](paritytech/litep2p#262))
- req-resp: Refactor to move functionality to dedicated methods
([#244](paritytech/litep2p#244))
- transport_service: Improve logs and move code from tokio::select macro
([#254](paritytech/litep2p#254))

- tcp/websocket/quic: Fix cancel memory leak
([#272](paritytech/litep2p#272))
- transport: Fix pending dials memory leak
([#271](paritytech/litep2p#271))
- ping: Fix memory leak of unremoved `pending_opens`
([#274](paritytech/litep2p#274))
- identify: Fix memory leak of unused `pending_opens`
([#273](paritytech/litep2p#273))
- kad: Fix not retrieving local records
([#221](paritytech/litep2p#221))

See release changelog for more details:
https://github.com/paritytech/litep2p/releases/tag/v0.8.0

cc @paritytech/networking

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Dmitry Markin <dmitry@markin.tech>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
I5-enhancement An additional feature request. T1-FRAME This PR/Issue is related to core FRAME, the framework.
Projects
Runtime / FRAME
Status: Backlog
Milestone
No milestone
Development

No branches or pull requests
4 participants
@juangirini @kianenigma @ggwpez @the-right-joyce