Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revive sub-du #4341

Closed
wants to merge 6 commits into from
Closed

Revive sub-du #4341

wants to merge 6 commits into from

Conversation

dastansam
Copy link
Contributor

@dastansam dastansam commented May 1, 2024

Revive sub-du

basically completes paritytech/substrate#12439 and closes #449.

First commit initial version contains the latest version of the previous PR. All the changes were done in the next commit.

Some notable changes

  • removed remote-externalities dependency. Tool was using it only for rpc_client() anyway, so now it's directly using substrate-rpc-client
  • switched to state_storageKeysPaged instead of state_storageSize() because the latter times out (30 s) on System::Account and many other large storages
  • removed storage_pairs options and directly counting the number of keys with paginated state read.
  • just some general cleanup and refactoring
Example output.
Scraping at block Some(0xbf9cf47882b314e5814ec803572079ce3a19898e7fd69642fbaaf4edbb33fcd0) of polkadot(1002000)
101 M │ │─┬ System
101 M │ │ │─ Account => Map(106,026,240 bytes, 1325328 keys)
128 K │ │ │─ BlockHash => Map(131,104 bytes, 4097 keys)
22  K │ │ │─ Events => Value(23,011 bytes)
156 B │ │ │─ Digest => Value(156 bytes)
32  B │ │ │─ ParentHash => Value(32 bytes)
18  B │ │ │─ BlockWeight => Value(18 bytes)
13  B │ │ │─ LastRuntimeUpgrade => Value(13 bytes)
4   B │ │ │─ EventCount => Value(4 bytes)
4   B │ │ │─ Number => Value(4 bytes)
1   B │ │ │─ UpgradedToTripleRefCount => Value(1 bytes)
1   B │ │ │─ UpgradedToU32RefCount => Value(1 bytes)
0   B │ │ │─ AuthorizedUpgrade => Value(0 bytes)
0   B │ │ │─ ExecutionPhase => Value(0 bytes)
0   B │ │ │─ EventTopics => Map(0 bytes, 0 keys)
0   B │ │ │─ ExtrinsicData => Map(0 bytes, 0 keys)
0   B │ │ │─ AllExtrinsicsLen => Value(0 bytes)
0   B │ │ │─ ExtrinsicCount => Value(0 bytes)
1   K │ │─┬ Scheduler
1   K │ │ │─ Agenda => Map(1,551 bytes, 1551 keys)
24  B │ │ │─ Lookup => Map(24 bytes, 3 keys)
0   B │ │ │─ IncompleteSince => Value(0 bytes)
37  K │ │─┬ Preimage
16  K │ │ │─ PreimageFor => Map(17,028 bytes, 396 keys)
11  K │ │ │─ StatusFor => Map(11,660 bytes, 220 keys)
9   K │ │ │─ RequestStatusFor => Map(9,646 bytes, 182 keys)
30  K │ │─┬ Babe
11  K │ │ │─ NextAuthorities => Value(11,882 bytes)
11  K │ │ │─ Authorities => Value(11,882 bytes)
7   K │ │ │─ UnderConstruction => Map(7,686 bytes, 3 keys)
33  B │ │ │─ AuthorVrfRandomness => Value(33 bytes)
32  B │ │ │─ NextRandomness => Value(32 bytes)
32  B │ │ │─ Randomness => Value(32 bytes)
17  B │ │ │─ NextEpochConfig => Value(17 bytes)
17  B │ │ │─ EpochConfig => Value(17 bytes)
8   B │ │ │─ EpochStart => Value(8 bytes)
8   B │ │ │─ CurrentSlot => Value(8 bytes)
8   B │ │ │─ GenesisSlot => Value(8 bytes)
8   B │ │ │─ EpochIndex => Value(8 bytes)
4   B │ │ │─ SegmentIndex => Value(4 bytes)
0   B │ │ │─ SkippedEpochs => Value(0 bytes)
0   B │ │ │─ Lateness => Value(0 bytes)
0   B │ │ │─ Initialized => Value(0 bytes)
0   B │ │ │─ PendingEpochConfigChange => Value(0 bytes)
8   B │ │─┬ Timestamp
8   B │ │ │─ Now => Value(8 bytes)
0   B │ │ │─ DidUpdate => Value(0 bytes)
170 K │ │─┬ Indices
170 K │ │ │─ Accounts => Map(174,881 bytes, 3569 keys)
1   M │ │─┬ Balances
1   M │ │ │─ Locks => Map(1,744,054 bytes, 67079 keys)
4   K │ │ │─ Freezes => Map(4,313 bytes, 227 keys)
3   K │ │ │─ Holds => Map(3,154 bytes, 166 keys)
16  B │ │ │─ InactiveIssuance => Value(16 bytes)
16  B │ │ │─ TotalIssuance => Value(16 bytes)
0   B │ │ │─ Reserves => Map(0 bytes, 0 keys)
0   B │ │ │─ Account => Map(0 bytes, 0 keys)
17  B │ │─┬ TransactionPayment
16  B │ │ │─ NextFeeMultiplier => Value(16 bytes)
1   B │ │ │─ StorageVersion => Value(1 bytes)
0   B │ │─┬ Authorship
0   B │ │ │─ Author => Value(0 bytes)
358 M │ │─┬ Staking
156 M │ │ │─ ErasStakersClipped => Map(164,055,375 bytes, 22275 keys)
156 M │ │ │─ ErasStakers => Map(164,055,375 bytes, 22275 keys)
23  M │ │ │─ Ledger => Map(24,802,085 bytes, 64421 keys)
17  M │ │ │─ ErasStakersPaged => Map(17,847,220 bytes, 2980 keys)
1   M │ │ │─ Bonded => Map(2,061,568 bytes, 64424 keys)
1   M │ │ │─ Nominators => Map(1,432,676 bytes, 37702 keys)
888 K │ │ │─ ErasRewardPoints => Map(909,330 bytes, 85 keys)
123 K │ │ │─ ErasValidatorPrefs => Map(126,225 bytes, 25245 keys)
66  K │ │ │─ ErasStakersOverview => Map(68,310 bytes, 2970 keys)
62  K │ │ │─ Payee => Map(64,424 bytes, 64424 keys)
14  K │ │ │─ ClaimedRewards => Map(14,385 bytes, 2877 keys)
9   K │ │ │─ SlashingSpans => Map(9,765 bytes, 465 keys)
7   K │ │ │─ SpanSlash => Map(7,936 bytes, 248 keys)
4   K │ │ │─ Validators => Map(4,940 bytes, 988 keys)
1   K │ │ │─ ErasTotalStake => Map(1,360 bytes, 85 keys)
1   K │ │ │─ ErasValidatorReward => Map(1,344 bytes, 84 keys)
340 B │ │ │─ ErasStartSessionIndex => Map(340 bytes, 85 keys)
233 B │ │ │─ BondedEras => Value(233 bytes)
16  B │ │ │─ CanceledSlashPayout => Value(16 bytes)
16  B │ │ │─ MinimumActiveStake => Value(16 bytes)
16  B │ │ │─ MinValidatorBond => Value(16 bytes)
16  B │ │ │─ MinNominatorBond => Value(16 bytes)
13  B │ │ │─ ActiveEra => Value(13 bytes)
4   B │ │ │─ CurrentPlannedSession => Value(4 bytes)
4   B │ │ │─ SlashRewardFraction => Value(4 bytes)
4   B │ │ │─ CurrentEra => Value(4 bytes)
4   B │ │ │─ CounterForNominators => Value(4 bytes)
4   B │ │ │─ MaxValidatorsCount => Value(4 bytes)
4   B │ │ │─ CounterForValidators => Value(4 bytes)
4   B │ │ │─ MinCommission => Value(4 bytes)
4   B │ │ │─ MinimumValidatorCount => Value(4 bytes)
4   B │ │ │─ ValidatorCount => Value(4 bytes)
1   B │ │ │─ ChillThreshold => Value(1 bytes)
1   B │ │ │─ Invulnerables => Value(1 bytes)
0   B │ │ │─ OffendingValidators => Value(0 bytes)
0   B │ │ │─ NominatorSlashInEra => Map(0 bytes, 0 keys)
0   B │ │ │─ ValidatorSlashInEra => Map(0 bytes, 0 keys)
0   B │ │ │─ UnappliedSlashes => Map(0 bytes, 0 keys)
0   B │ │ │─ ForceEra => Value(0 bytes)
0   B │ │ │─ MaxNominatorsCount => Value(0 bytes)
564 K │ │─┬ Offences
509 K │ │ │─ Reports => Map(522,004 bytes, 2534 keys)
54  K │ │ │─ ConcurrentReportsIndex => Map(55,671 bytes, 1687 keys)
6   K │ │─┬ Historical
6   K │ │ │─ HistoricalSessions => Map(6,192 bytes, 172 keys)
8   B │ │ │─ StoredRange => Value(8 bytes)
905 K │ │─┬ Session
427 K │ │ │─ KeyOwner => Map(437,600 bytes, 13675 keys)
403 K │ │ │─ NextKeys => Map(413,406 bytes, 2142 keys)
65  K │ │ │─ QueuedKeys => Value(66,827 bytes)
9   K │ │ │─ Validators => Value(9,506 bytes)
4   B │ │ │─ CurrentIndex => Value(4 bytes)
1   B │ │ │─ QueuedChanged => Value(1 bytes)
0   B │ │ │─ DisabledValidators => Value(0 bytes)
12  K │ │─┬ Grandpa
11  K │ │ │─ Authorities => Value(11,882 bytes)
672 B │ │ │─ SetIdSession => Map(672 bytes, 168 keys)
8   B │ │ │─ CurrentSetId => Value(8 bytes)
0   B │ │ │─ Stalled => Value(0 bytes)
0   B │ │ │─ NextForced => Value(0 bytes)
0   B │ │ │─ PendingChange => Value(0 bytes)
0   B │ │ │─ State => Value(0 bytes)
18  K │ │─┬ AuthorityDiscovery
9   K │ │ │─ NextKeys => Value(9,506 bytes)
9   K │ │ │─ Keys => Value(9,506 bytes)
2   K │ │─┬ Treasury
2   K │ │ │─ Proposals => Map(2,688 bytes, 28 keys)
89  B │ │ │─ Approvals => Value(89 bytes)
16  B │ │ │─ Deactivated => Value(16 bytes)
4   B │ │ │─ ProposalCount => Value(4 bytes)
0   B │ │ │─ Spends => Map(0 bytes, 0 keys)
0   B │ │ │─ SpendCount => Value(0 bytes)
1   M │ │─┬ ConvictionVoting
1   M │ │ │─ VotingFor => Map(1,624,652 bytes, 21377 keys)
288 K │ │ │─ ClassLocksFor => Map(295,845 bytes, 5379 keys)
154 K │ │─┬ Referenda
154 K │ │ │─ ReferendumInfoFor => Map(157,896 bytes, 731 keys)
64  B │ │ │─ DecidingCount => Map(64 bytes, 16 keys)
21  B │ │ │─ TrackQueue => Map(21 bytes, 1 keys)
4   B │ │ │─ ReferendumCount => Value(4 bytes)
0   B │ │ │─ MetadataOf => Map(0 bytes, 0 keys)
0   B │ │─┬ Whitelist
0   B │ │ │─ WhitelistedCall => Map(0 bytes, 8 keys)
11  K │ │─┬ Claims
5   K │ │ │─ Claims => Map(5,808 bytes, 363 keys)
4   K │ │ │─ Preclaims => Map(4,420 bytes, 221 keys)
1   K │ │ │─ Vesting => Map(1,224 bytes, 34 keys)
385 B │ │ │─ Signing => Map(385 bytes, 385 keys)
16  B │ │ │─ Total => Value(16 bytes)
29  K │ │─┬ Vesting
29  K │ │ │─ Vesting => Map(29,896 bytes, 808 keys)
1   B │ │ │─ StorageVersion => Value(1 bytes)
290 K │ │─┬ Identity
181 K │ │ │─ IdentityOf => Map(185,818 bytes, 1753 keys)
80  K │ │ │─ SubsOf => Map(82,902 bytes, 246 keys)
27  K │ │ │─ SuperOf => Map(28,423 bytes, 661 keys)
229 B │ │ │─ Registrars => Value(229 bytes)
0   B │ │ │─ PendingUsernames => Map(0 bytes, 0 keys)
0   B │ │ │─ AccountOfUsername => Map(0 bytes, 0 keys)
0   B │ │ │─ UsernameAuthorities => Map(0 bytes, 0 keys)
80  K │ │─┬ Proxy
79  K │ │ │─ Proxies => Map(81,702 bytes, 1513 keys)
595 B │ │ │─ Announcements => Map(595 bytes, 7 keys)
10  K │ │─┬ Multisig
10  K │ │ │─ Multisigs => Map(10,648 bytes, 88 keys)
5   K │ │─┬ Bounties
2   K │ │ │─ BountyDescriptions => Map(2,730 bytes, 26 keys)
2   K │ │ │─ Bounties => Map(2,522 bytes, 26 keys)
5   B │ │ │─ BountyApprovals => Value(5 bytes)
4   B │ │ │─ BountyCount => Value(4 bytes)
9   K │ │─┬ ChildBounties
7   K │ │ │─ ChildBounties => Map(8,107 bytes, 67 keys)
1005B │ │ │─ ChildBountyDescriptions => Map(1,005 bytes, 67 keys)
208 B │ │ │─ ChildrenCuratorFees => Map(208 bytes, 13 keys)
52  B │ │ │─ ParentChildBounties => Map(52 bytes, 13 keys)
4   B │ │ │─ ChildBountyCount => Value(4 bytes)
53  B │ │─┬ ElectionProviderMultiPhase
48  B │ │ │─ MinimumUntrustedScore => Value(48 bytes)
4   B │ │ │─ Round => Value(4 bytes)
1   B │ │ │─ CurrentPhase => Value(1 bytes)
0   B │ │ │─ SignedSubmissionsMap => Map(0 bytes, 0 keys)
0   B │ │ │─ SignedSubmissionIndices => Value(0 bytes)
0   B │ │ │─ SignedSubmissionNextIndex => Value(0 bytes)
0   B │ │ │─ SnapshotMetadata => Value(0 bytes)
0   B │ │ │─ DesiredTargets => Value(0 bytes)
0   B │ │ │─ Snapshot => Value(0 bytes)
0   B │ │ │─ QueuedSolution => Value(0 bytes)
4   M │ │─┬ VoterList
4   M │ │ │─ ListNodes => Map(4,410,660 bytes, 38690 keys)
9   K │ │ │─ ListBags => Map(10,098 bytes, 153 keys)
4   B │ │ │─ CounterForListNodes => Value(4 bytes)
2   M │ │─┬ NominationPools
2   M │ │ │─ PoolMembers => Map(2,708,013 bytes, 47509 keys)
34  K │ │ │─ BondedPools => Map(35,639 bytes, 227 keys)
17  K │ │ │─ RewardPools => Map(18,160 bytes, 227 keys)
13  K │ │ │─ SubPoolsStorage => Map(13,869 bytes, 201 keys)
9   K │ │ │─ ClaimPermissions => Map(9,958 bytes, 9958 keys)
908 B │ │ │─ ReversePoolIdLookup => Map(908 bytes, 227 keys)
908 B │ │ │─ Metadata => Map(908 bytes, 227 keys)
16  B │ │ │─ MinCreateBond => Value(16 bytes)
16  B │ │ │─ MinJoinBond => Value(16 bytes)
16  B │ │ │─ TotalValueLocked => Value(16 bytes)
4   B │ │ │─ CounterForReversePoolIdLookup => Value(4 bytes)
4   B │ │ │─ LastPoolId => Value(4 bytes)
4   B │ │ │─ CounterForMetadata => Value(4 bytes)
4   B │ │ │─ CounterForSubPoolsStorage => Value(4 bytes)
4   B │ │ │─ CounterForRewardPools => Value(4 bytes)
4   B │ │ │─ CounterForBondedPools => Value(4 bytes)
4   B │ │ │─ CounterForPoolMembers => Value(4 bytes)
4   B │ │ │─ GlobalMaxCommission => Value(4 bytes)
0   B │ │ │─ MaxPoolMembersPerPool => Value(0 bytes)
0   B │ │ │─ MaxPoolMembers => Value(0 bytes)
0   B │ │ │─ MaxPools => Value(0 bytes)
8   B │ │─┬ FastUnstake
4   B │ │ │─ ErasToCheckPerBlock => Value(4 bytes)
4   B │ │ │─ CounterForQueue => Value(4 bytes)
0   B │ │ │─ Queue => Map(0 bytes, 0 keys)
0   B │ │ │─ Head => Value(0 bytes)
238 B │ │─┬ Configuration
237 B │ │ │─ ActiveConfig => Value(237 bytes)
1   B │ │ │─ PendingConfigs => Value(1 bytes)
0   B │ │ │─ BypassConsistencyCheck => Value(0 bytes)
10  K │ │─┬ ParasShared
9   K │ │ │─ ActiveValidatorKeys => Value(9,506 bytes)
1   K │ │ │─ ActiveValidatorIndices => Value(1,190 bytes)
69  B │ │ │─ AllowedRelayParents => Value(69 bytes)
4   B │ │ │─ CurrentSessionIndex => Value(4 bytes)
20  K │ │─┬ ParaInclusion
11  K │ │ │─ PendingAvailability => Map(11,760 bytes, 28 keys)
5   K │ │ │─ PendingAvailabilityCommitments => Map(5,572 bytes, 28 keys)
3   K │ │ │─ AvailabilityBitfields => Map(3,861 bytes, 297 keys)
18  K │ │─┬ ParaInherent
18  K │ │ │─ OnChainVotes => Value(18,766 bytes)
0   B │ │ │─ Included => Value(0 bytes)
2   K │ │─┬ ParaScheduler
1   K │ │ │─ ValidatorGroups => Value(1,248 bytes)
537 B │ │ │─ ClaimQueue => Value(537 bytes)
424 B │ │ │─ AvailabilityCores => Value(424 bytes)
4   B │ │ │─ SessionStartBlock => Value(4 bytes)
39  M │ │─┬ Paras
39  M │ │ │─ CodeByHash => Map(41,849,956 bytes, 71 keys)
12  K │ │ │─ Heads => Map(13,160 bytes, 70 keys)
2   K │ │ │─ CurrentCodeHash => Map(2,240 bytes, 70 keys)
284 B │ │ │─ CodeByHashRefs => Map(284 bytes, 71 keys)
270 B │ │ │─ PastCodeMeta => Map(270 bytes, 45 keys)
208 B │ │ │─ MostRecentContext => Map(208 bytes, 52 keys)
201 B │ │ │─ Parachains => Value(201 bytes)
70  B │ │ │─ ParaLifecycles => Map(70 bytes, 70 keys)
32  B │ │ │─ FutureCodeHash => Map(32 bytes, 1 keys)
4   B │ │ │─ FutureCodeUpgrades => Map(4 bytes, 1 keys)
1   B │ │ │─ UpcomingUpgrades => Value(1 bytes)
1   B │ │ │─ UpgradeCooldowns => Value(1 bytes)
1   B │ │ │─ UpgradeGoAheadSignal => Map(1 bytes, 1 keys)
1   B │ │ │─ PastCodePruning => Value(1 bytes)
1   B │ │ │─ PvfActiveVoteList => Value(1 bytes)
0   B │ │ │─ UpcomingParasGenesis => Map(0 bytes, 0 keys)
0   B │ │ │─ ActionsQueue => Map(0 bytes, 0 keys)
0   B │ │ │─ UpgradeRestrictionSignal => Map(0 bytes, 0 keys)
0   B │ │ │─ PastCodeHash => Map(0 bytes, 0 keys)
0   B │ │ │─ PvfActiveVoteMap => Map(0 bytes, 0 keys)
0   B │ │─┬ Initializer
0   B │ │ │─ BufferedSessionChanges => Value(0 bytes)
0   B │ │ │─ HasInitialized => Value(0 bytes)
2   K │ │─┬ Dmp
1   K │ │ │─ DownwardMessageQueueHeads => Map(1,472 bytes, 46 keys)
784 B │ │ │─ DeliveryFeeFactor => Map(784 bytes, 49 keys)
56  B │ │ │─ DownwardMessageQueues => Map(56 bytes, 56 keys)
21  K │ │─┬ Hrmp
18  K │ │ │─ HrmpChannels => Map(18,700 bytes, 220 keys)
777 B │ │ │─ HrmpEgressChannelsIndex => Map(777 bytes, 37 keys)
756 B │ │ │─ HrmpIngressChannelsIndex => Map(756 bytes, 36 keys)
596 B │ │ │─ HrmpChannelContents => Map(596 bytes, 4 keys)
462 B │ │ │─ HrmpOpenChannelRequests => Map(462 bytes, 14 keys)
204 B │ │ │─ HrmpWatermarks => Map(204 bytes, 51 keys)
113 B │ │ │─ HrmpOpenChannelRequestsList => Value(113 bytes)
51  B │ │ │─ HrmpChannelDigests => Map(51 bytes, 51 keys)
48  B │ │ │─ HrmpOpenChannelRequestCount => Map(48 bytes, 12 keys)
0   B │ │ │─ HrmpCloseChannelRequestsList => Value(0 bytes)
0   B │ │ │─ HrmpCloseChannelRequests => Map(0 bytes, 0 keys)
0   B │ │ │─ HrmpAcceptedChannelRequestCount => Map(0 bytes, 0 keys)
286 K │ │─┬ ParaSessionInfo
212 K │ │ │─ Sessions => Map(217,112 bytes, 7 keys)
64  K │ │ │─ AccountKeys => Map(66,542 bytes, 7 keys)
9   K │ │ │─ AssignmentKeysUnsafe => Value(9,506 bytes)
182 B │ │ │─ SessionExecutorParams => Map(182 bytes, 7 keys)
4   B │ │ │─ EarliestStoredSession => Value(4 bytes)
1   M │ │─┬ ParasDisputes
1   M │ │ │─ Included => Map(1,155,544 bytes, 288886 keys)
4   B │ │ │─ LastPrunedSession => Value(4 bytes)
0   B │ │ │─ Frozen => Value(0 bytes)
0   B │ │ │─ BackersOnDisputes => Map(0 bytes, 0 keys)
0   B │ │ │─ Disputes => Map(0 bytes, 0 keys)
0   B │ │─┬ ParasSlashing
0   B │ │ │─ ValidatorSetCounts => Map(0 bytes, 0 keys)
0   B │ │ │─ UnappliedSlashes => Map(0 bytes, 0 keys)
4   K │ │─┬ Registrar
4   K │ │ │─ Paras => Map(4,650 bytes, 93 keys)
4   B │ │ │─ NextFreeParaId => Value(4 bytes)
4   B │ │ │─ PendingSwap => Map(4 bytes, 1 keys)
7   K │ │─┬ Slots
7   K │ │ │─ Leases => Map(7,992 bytes, 54 keys)
4   B │ │─┬ Auctions
4   B │ │ │─ AuctionCounter => Value(4 bytes)
0   B │ │ │─ Winning => Map(0 bytes, 0 keys)
0   B │ │ │─ ReservedAmounts => Map(0 bytes, 0 keys)
0   B │ │ │─ AuctionInfo => Value(0 bytes)
3   K │ │─┬ Crowdloan
3   K │ │ │─ Funds => Map(3,876 bytes, 38 keys)
5   B │ │ │─ NewRaise => Value(5 bytes)
4   B │ │ │─ NextFundIndex => Value(4 bytes)
4   B │ │ │─ EndingsCount => Value(4 bytes)
14  B │ │─┬ StateTrieMigration
14  B │ │ │─ MigrationProcess => Value(14 bytes)
0   B │ │ │─ SignedMigrationMaxLimits => Value(0 bytes)
0   B │ │ │─ AutoLimits => Value(0 bytes)
2   K │ │─┬ XcmPallet
1   K │ │ │─ AssetTraps => Map(1,656 bytes, 414 keys)
392 B │ │ │─ VersionNotifyTargets => Map(392 bytes, 28 keys)
320 B │ │ │─ VersionNotifiers => Map(320 bytes, 40 keys)
320 B │ │ │─ Queries => Map(320 bytes, 40 keys)
116 B │ │ │─ SupportedVersion => Map(116 bytes, 29 keys)
8   B │ │ │─ QueryCounter => Value(8 bytes)
4   B │ │ │─ SafeXcmVersion => Value(4 bytes)
1   B │ │ │─ VersionDiscoveryQueue => Value(1 bytes)
0   B │ │ │─ XcmExecutionSuspended => Value(0 bytes)
0   B │ │ │─ LockedFungibles => Map(0 bytes, 0 keys)
0   B │ │ │─ RemoteLockedFungibles => Map(0 bytes, 0 keys)
0   B │ │ │─ CurrentMigration => Value(0 bytes)
928 B │ │─┬ MessageQueue
928 B │ │ │─ BookStateFor => Map(928 bytes, 32 keys)
0   B │ │ │─ Pages => Map(0 bytes, 0 keys)
0   B │ │ │─ ServiceHead => Value(0 bytes)
0   B │ │─┬ AssetRate
0   B │ │ │─ ConversionRateToNative => Map(0 bytes, 0 keys)
19  K │ │─┬ Beefy
9   K │ │ │─ NextAuthorities => Value(9,803 bytes)
9   K │ │ │─ Authorities => Value(9,803 bytes)
672 B │ │ │─ SetIdSession => Map(672 bytes, 168 keys)
8   B │ │ │─ ValidatorSetId => Value(8 bytes)
5   B │ │ │─ GenesisBlock => Value(5 bytes)
424 B │ │─┬ Mmr
384 B │ │ │─ Nodes => Map(384 bytes, 12 keys)
32  B │ │ │─ RootHash => Value(32 bytes)
8   B │ │ │─ NumberOfLeaves => Value(8 bytes)
88  B │ │─┬ BeefyMmrLeaf
44  B │ │ │─ BeefyNextAuthorities => Value(44 bytes)
44  B │ │ │─ BeefyAuthorities => Value(44 bytes)

Total size: 513 M polkadot
822.7441s

polkadot: 16FqwPZ8GRC5U5D4Fu7W33nA55ZXzXGWHwmbnE1eT6pxuqcT

cc @kianenigma @ggwpez

Comment on lines +246 to +251
// don't really need to read for all, just for the first one to get the size.
if let Some(size) = value_size {
sum += size;
} else if let Ok(Some(value)) = client.storage(key.clone(), at).await {
value_size = Some(value.0.len() as u64);
sum += value.0.len() as u64;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not 100% sure, but it should be possible that the value size is dynamic, right? for example, if Option<T> is stored or if value is unbounded. however, it should be good enough solution for most of the most populated storages, imo
cc @ggwpez

@dastansam dastansam marked this pull request as ready for review May 1, 2024 02:02
@dastansam dastansam force-pushed the feat/sub-du branch 5 times, most recently from 5466fca to 63f54f3 Compare May 1, 2024 11:27
@dastansam
Copy link
Contributor Author

kusama output:
Scraping at block Some(0xae81611de99b6528952bf2e23c94d33dab206e49f73455946f635956eeea4a85) of kusama(1002000)
24  M │ │─┬ System

24  M │ │ │─ Account => Map(25,629,520 bytes, 320369 keys)
135 K │ │ │─ BlockHash => Map(139,136 bytes, 4348 keys)
22  K │ │ │─ Events => Value(23,065 bytes)
156 B │ │ │─ Digest => Value(156 bytes)
32  B │ │ │─ ParentHash => Value(32 bytes)
14  B │ │ │─ BlockWeight => Value(14 bytes)
11  B │ │ │─ LastRuntimeUpgrade => Value(11 bytes)
4   B │ │ │─ EventCount => Value(4 bytes)
4   B │ │ │─ Number => Value(4 bytes)
1   B │ │ │─ UpgradedToTripleRefCount => Value(1 bytes)
1   B │ │ │─ UpgradedToU32RefCount => Value(1 bytes)
0   B │ │ │─ AuthorizedUpgrade => Value(0 bytes)
0   B │ │ │─ ExecutionPhase => Value(0 bytes)
0   B │ │ │─ EventTopics => Map(0 bytes, 0 keys)
0   B │ │ │─ ExtrinsicData => Map(0 bytes, 0 keys)
0   B │ │ │─ AllExtrinsicsLen => Value(0 bytes)
0   B │ │ │─ ExtrinsicCount => Value(0 bytes)
81  K │ │─┬ Babe

39  K │ │ │─ NextAuthorities => Value(40,002 bytes)
39  K │ │ │─ Authorities => Value(40,002 bytes)
2   K │ │ │─ UnderConstruction => Map(2,787 bytes, 3 keys)
33  B │ │ │─ AuthorVrfRandomness => Value(33 bytes)
32  B │ │ │─ NextRandomness => Value(32 bytes)
32  B │ │ │─ Randomness => Value(32 bytes)
17  B │ │ │─ NextEpochConfig => Value(17 bytes)
17  B │ │ │─ EpochConfig => Value(17 bytes)
8   B │ │ │─ EpochStart => Value(8 bytes)
8   B │ │ │─ CurrentSlot => Value(8 bytes)
8   B │ │ │─ GenesisSlot => Value(8 bytes)
8   B │ │ │─ EpochIndex => Value(8 bytes)
4   B │ │ │─ SegmentIndex => Value(4 bytes)
0   B │ │ │─ SkippedEpochs => Value(0 bytes)
0   B │ │ │─ Lateness => Value(0 bytes)
0   B │ │ │─ Initialized => Value(0 bytes)
0   B │ │ │─ PendingEpochConfigChange => Value(0 bytes)
8   B │ │─┬ Timestamp

8   B │ │ │─ Now => Value(8 bytes)
0   B │ │ │─ DidUpdate => Value(0 bytes)
56  K │ │─┬ Indices

56  K │ │ │─ Accounts => Map(57,428 bytes, 1172 keys)
697 K │ │─┬ Balances

693 K │ │ │─ Locks => Map(710,190 bytes, 27315 keys)
3   K │ │ │─ Freezes => Map(3,458 bytes, 182 keys)
627 B │ │ │─ Holds => Map(627 bytes, 33 keys)
16  B │ │ │─ InactiveIssuance => Value(16 bytes)
16  B │ │ │─ TotalIssuance => Value(16 bytes)
0   B │ │ │─ Reserves => Map(0 bytes, 0 keys)
0   B │ │ │─ Account => Map(0 bytes, 0 keys)
17  B │ │─┬ TransactionPayment

16  B │ │ │─ NextFeeMultiplier => Value(16 bytes)
1   B │ │ │─ StorageVersion => Value(1 bytes)
0   B │ │─┬ Authorship

0   B │ │ │─ Author => Value(0 bytes)
30  M │ │─┬ Staking

9   M │ │ │─ Ledger => Map(10,003,326 bytes, 26394 keys)
5   M │ │ │─ ErasStakersClipped => Map(5,344,000 bytes, 32000 keys)
5   M │ │ │─ ErasStakers => Map(5,344,000 bytes, 32000 keys)
4   M │ │ │─ ErasStakersPaged => Map(4,663,824 bytes, 52998 keys)
2   M │ │ │─ ErasRewardPoints => Map(3,045,210 bytes, 85 keys)
879 K │ │ │─ ErasStakersOverview => Map(901,000 bytes, 53000 keys)
824 K │ │ │─ Bonded => Map(844,704 bytes, 26397 keys)
581 K │ │ │─ Nominators => Map(595,536 bytes, 15672 keys)
415 K │ │ │─ ErasValidatorPrefs => Map(425,000 bytes, 85000 keys)
250 K │ │ │─ ClaimedRewards => Map(256,690 bytes, 51338 keys)
78  K │ │ │─ SpanSlash => Map(80,512 bytes, 2516 keys)
63  K │ │ │─ SlashingSpans => Map(64,855 bytes, 3815 keys)
25  K │ │ │─ Payee => Map(26,397 bytes, 26397 keys)
10  K │ │ │─ Validators => Map(10,905 bytes, 2181 keys)
1   K │ │ │─ ErasTotalStake => Map(1,360 bytes, 85 keys)
1   K │ │ │─ ErasValidatorReward => Map(1,344 bytes, 84 keys)
340 B │ │ │─ ErasStartSessionIndex => Map(340 bytes, 85 keys)
233 B │ │ │─ BondedEras => Value(233 bytes)
16  B │ │ │─ CanceledSlashPayout => Value(16 bytes)
16  B │ │ │─ MinimumActiveStake => Value(16 bytes)
16  B │ │ │─ MinValidatorBond => Value(16 bytes)
16  B │ │ │─ MinNominatorBond => Value(16 bytes)
13  B │ │ │─ ActiveEra => Value(13 bytes)
4   B │ │ │─ CurrentPlannedSession => Value(4 bytes)
4   B │ │ │─ SlashRewardFraction => Value(4 bytes)
4   B │ │ │─ CurrentEra => Value(4 bytes)
4   B │ │ │─ MaxNominatorsCount => Value(4 bytes)
4   B │ │ │─ CounterForNominators => Value(4 bytes)
4   B │ │ │─ MaxValidatorsCount => Value(4 bytes)
4   B │ │ │─ CounterForValidators => Value(4 bytes)
4   B │ │ │─ MinCommission => Value(4 bytes)
4   B │ │ │─ MinimumValidatorCount => Value(4 bytes)
4   B │ │ │─ ValidatorCount => Value(4 bytes)
1   B │ │ │─ ForceEra => Value(1 bytes)
1   B │ │ │─ Invulnerables => Value(1 bytes)
0   B │ │ │─ ChillThreshold => Value(0 bytes)
0   B │ │ │─ OffendingValidators => Value(0 bytes)
0   B │ │ │─ NominatorSlashInEra => Map(0 bytes, 0 keys)
0   B │ │ │─ ValidatorSlashInEra => Map(0 bytes, 0 keys)
0   B │ │ │─ UnappliedSlashes => Map(0 bytes, 0 keys)
4   M │ │─┬ Offences

3   M │ │ │─ Reports => Map(3,840,528 bytes, 43152 keys)
638 K │ │ │─ ConcurrentReportsIndex => Map(653,565 bytes, 19805 keys)
6   K │ │─┬ Historical

6   K │ │ │─ HistoricalSessions => Map(6,228 bytes, 173 keys)
8   B │ │ │─ StoredRange => Value(8 bytes)
2   M │ │─┬ Session

980 K │ │ │─ KeyOwner => Map(1,003,808 bytes, 31369 keys)
842 K │ │ │─ NextKeys => Map(863,096 bytes, 4472 keys)
219 K │ │ │─ QueuedKeys => Value(225,002 bytes)
31  K │ │ │─ Validators => Value(32,002 bytes)
4   B │ │ │─ CurrentIndex => Value(4 bytes)
1   B │ │ │─ QueuedChanged => Value(1 bytes)
0   B │ │ │─ DisabledValidators => Value(0 bytes)
39  K │ │─┬ Grandpa

39  K │ │ │─ Authorities => Value(40,002 bytes)
672 B │ │ │─ SetIdSession => Map(672 bytes, 168 keys)
8   B │ │ │─ CurrentSetId => Value(8 bytes)
0   B │ │ │─ Stalled => Value(0 bytes)
0   B │ │ │─ NextForced => Value(0 bytes)
0   B │ │ │─ PendingChange => Value(0 bytes)
0   B │ │ │─ State => Value(0 bytes)
62  K │ │─┬ AuthorityDiscovery

31  K │ │ │─ NextKeys => Value(32,002 bytes)
31  K │ │ │─ Keys => Value(32,002 bytes)
117 B │ │─┬ Treasury

96  B │ │ │─ Proposals => Map(96 bytes, 1 keys)
16  B │ │ │─ Deactivated => Value(16 bytes)
4   B │ │ │─ ProposalCount => Value(4 bytes)
1   B │ │ │─ Approvals => Value(1 bytes)
0   B │ │ │─ Spends => Map(0 bytes, 0 keys)
0   B │ │ │─ SpendCount => Value(0 bytes)
2   M │ │─┬ ConvictionVoting

2   M │ │ │─ VotingFor => Map(2,809,560 bytes, 23413 keys)
288 K │ │ │─ ClassLocksFor => Map(295,295 bytes, 3245 keys)
2   K │ │─┬ Referenda

2   K │ │ │─ ReferendumInfoFor => Map(2,695 bytes, 385 keys)
52  B │ │ │─ DecidingCount => Map(52 bytes, 13 keys)
4   B │ │ │─ ReferendumCount => Value(4 bytes)
1   B │ │ │─ TrackQueue => Map(1 bytes, 1 keys)
0   B │ │ │─ MetadataOf => Map(0 bytes, 0 keys)
7   K │ │─┬ FellowshipCollective

4   K │ │ │─ IndexToId => Map(4,960 bytes, 155 keys)
1   K │ │ │─ Voting => Map(1,875 bytes, 375 keys)
620 B │ │ │─ IdToIndex => Map(620 bytes, 155 keys)
94  B │ │ │─ Members => Map(94 bytes, 47 keys)
28  B │ │ │─ MemberCount => Map(28 bytes, 7 keys)
0   B │ │ │─ VotingCleanup => Map(0 bytes, 0 keys)
538 B │ │─┬ FellowshipReferenda

518 B │ │ │─ ReferendumInfoFor => Map(518 bytes, 74 keys)
16  B │ │ │─ DecidingCount => Map(16 bytes, 4 keys)
4   B │ │ │─ ReferendumCount => Value(4 bytes)
0   B │ │ │─ MetadataOf => Map(0 bytes, 0 keys)
0   B │ │ │─ TrackQueue => Map(0 bytes, 0 keys)
0   B │ │─┬ Whitelist

0   B │ │ │─ WhitelistedCall => Map(0 bytes, 3 keys)
13  K │ │─┬ Claims

13  K │ │ │─ Claims => Map(14,048 bytes, 878 keys)
16  B │ │ │─ Total => Value(16 bytes)
0   B │ │ │─ Preclaims => Map(0 bytes, 0 keys)
0   B │ │ │─ Signing => Map(0 bytes, 0 keys)
0   B │ │ │─ Vesting => Map(0 bytes, 0 keys)
344 K │ │─┬ Identity

258 K │ │ │─ IdentityOf => Map(264,926 bytes, 4273 keys)
58  K │ │ │─ SuperOf => Map(60,060 bytes, 1540 keys)
27  K │ │ │─ SubsOf => Map(27,685 bytes, 565 keys)
343 B │ │ │─ Registrars => Value(343 bytes)
0   B │ │ │─ PendingUsernames => Map(0 bytes, 0 keys)
0   B │ │ │─ AccountOfUsername => Map(0 bytes, 0 keys)
0   B │ │ │─ UsernameAuthorities => Map(0 bytes, 0 keys)
11  K │ │─┬ Society

5   K │ │ │─ MemberByIndex => Map(5,216 bytes, 163 keys)
2   K │ │ │─ Members => Map(2,119 bytes, 163 keys)
1   K │ │ │─ Bids => Value(1,333 bytes)
1   K │ │ │─ Votes => Map(1,250 bytes, 250 keys)
1   K │ │ │─ Payouts => Map(1,122 bytes, 66 keys)
260 B │ │ │─ DefenderVotes => Map(260 bytes, 52 keys)
221 B │ │ │─ SuspendedMembers => Map(221 bytes, 17 keys)
184 B │ │ │─ Candidates => Map(184 bytes, 4 keys)
72  B │ │ │─ Defending => Value(72 bytes)
32  B │ │ │─ Skeptic => Value(32 bytes)
32  B │ │ │─ Rules => Value(32 bytes)
32  B │ │ │─ Head => Value(32 bytes)
32  B │ │ │─ Founder => Value(32 bytes)
28  B │ │ │─ Parameters => Value(28 bytes)
16  B │ │ │─ Pot => Value(16 bytes)
4   B │ │ │─ ChallengeRoundCount => Value(4 bytes)
4   B │ │ │─ RoundCount => Value(4 bytes)
4   B │ │ │─ MemberCount => Value(4 bytes)
0   B │ │ │─ NextHead => Value(0 bytes)
0   B │ │ │─ VoteClearCursor => Map(0 bytes, 0 keys)
6   K │ │─┬ Recovery

6   K │ │ │─ Recoverable => Map(6,435 bytes, 117 keys)
416 B │ │ │─ Proxy => Map(416 bytes, 13 keys)
273 B │ │ │─ ActiveRecoveries => Map(273 bytes, 13 keys)
334 B │ │─┬ Vesting

333 B │ │ │─ Vesting => Map(333 bytes, 9 keys)
1   B │ │ │─ StorageVersion => Value(1 bytes)
219 B │ │─┬ Scheduler

195 B │ │ │─ Agenda => Map(195 bytes, 13 keys)
24  B │ │ │─ Lookup => Map(24 bytes, 3 keys)
0   B │ │ │─ IncompleteSince => Value(0 bytes)
135 K │ │─┬ Proxy

134 K │ │ │─ Proxies => Map(138,138 bytes, 1518 keys)
510 B │ │ │─ Announcements => Map(510 bytes, 6 keys)
8   K │ │─┬ Multisig

8   K │ │ │─ Multisigs => Map(8,722 bytes, 98 keys)
17  K │ │─┬ Preimage

7   K │ │ │─ StatusFor => Map(7,791 bytes, 147 keys)
7   K │ │ │─ PreimageFor => Map(7,480 bytes, 187 keys)
2   K │ │ │─ RequestStatusFor => Map(2,279 bytes, 43 keys)
989 B │ │─┬ Bounties

776 B │ │ │─ Bounties => Map(776 bytes, 8 keys)
208 B │ │ │─ BountyDescriptions => Map(208 bytes, 8 keys)
4   B │ │ │─ BountyCount => Value(4 bytes)
1   B │ │ │─ BountyApprovals => Value(1 bytes)
3   K │ │─┬ ChildBounties

2   K │ │ │─ ChildBounties => Map(2,662 bytes, 22 keys)
418 B │ │ │─ ChildBountyDescriptions => Map(418 bytes, 22 keys)
128 B │ │ │─ ChildrenCuratorFees => Map(128 bytes, 8 keys)
36  B │ │ │─ ParentChildBounties => Map(36 bytes, 9 keys)
4   B │ │ │─ ChildBountyCount => Value(4 bytes)
53  B │ │─┬ ElectionProviderMultiPhase

48  B │ │ │─ MinimumUntrustedScore => Value(48 bytes)
4   B │ │ │─ Round => Value(4 bytes)
1   B │ │ │─ CurrentPhase => Value(1 bytes)
0   B │ │ │─ SignedSubmissionsMap => Map(0 bytes, 0 keys)
0   B │ │ │─ SignedSubmissionIndices => Value(0 bytes)
0   B │ │ │─ SignedSubmissionNextIndex => Value(0 bytes)
0   B │ │ │─ SnapshotMetadata => Value(0 bytes)
0   B │ │ │─ DesiredTargets => Value(0 bytes)
0   B │ │ │─ Snapshot => Value(0 bytes)
0   B │ │ │─ QueuedSolution => Value(0 bytes)
9   K │ │─┬ Nis

9   K │ │ │─ QueueTotals => Value(10,002 bytes)
1   B │ │ │─ Queues => Map(1 bytes, 1 keys)
0   B │ │ │─ Receipts => Map(0 bytes, 0 keys)
0   B │ │ │─ Summary => Value(0 bytes)
0   B │ │─┬ NisCounterpartBalances

0   B │ │ │─ Freezes => Map(0 bytes, 0 keys)
0   B │ │ │─ Holds => Map(0 bytes, 0 keys)
0   B │ │ │─ Reserves => Map(0 bytes, 0 keys)
0   B │ │ │─ Locks => Map(0 bytes, 0 keys)
0   B │ │ │─ Account => Map(0 bytes, 0 keys)
0   B │ │ │─ InactiveIssuance => Value(0 bytes)
0   B │ │ │─ TotalIssuance => Value(0 bytes)
1   M │ │─┬ VoterList

1   M │ │ │─ ListNodes => Map(2,035,242 bytes, 17853 keys)
10  K │ │ │─ ListBags => Map(10,956 bytes, 166 keys)
4   B │ │ │─ CounterForListNodes => Value(4 bytes)
158 K │ │─┬ NominationPools

106 K │ │ │─ PoolMembers => Map(109,446 bytes, 2958 keys)
27  K │ │ │─ BondedPools => Map(28,574 bytes, 182 keys)
14  K │ │ │─ RewardPools => Map(14,560 bytes, 182 keys)
4   K │ │ │─ Metadata => Map(4,368 bytes, 182 keys)
3   K │ │ │─ SubPoolsStorage => Map(3,993 bytes, 121 keys)
732 B │ │ │─ ClaimPermissions => Map(732 bytes, 732 keys)
728 B │ │ │─ ReversePoolIdLookup => Map(728 bytes, 182 keys)
16  B │ │ │─ MinCreateBond => Value(16 bytes)
16  B │ │ │─ MinJoinBond => Value(16 bytes)
16  B │ │ │─ TotalValueLocked => Value(16 bytes)
4   B │ │ │─ CounterForReversePoolIdLookup => Value(4 bytes)
4   B │ │ │─ LastPoolId => Value(4 bytes)
4   B │ │ │─ CounterForMetadata => Value(4 bytes)
4   B │ │ │─ CounterForSubPoolsStorage => Value(4 bytes)
4   B │ │ │─ CounterForRewardPools => Value(4 bytes)
4   B │ │ │─ CounterForBondedPools => Value(4 bytes)
4   B │ │ │─ CounterForPoolMembers => Value(4 bytes)
4   B │ │ │─ GlobalMaxCommission => Value(4 bytes)
0   B │ │ │─ MaxPoolMembersPerPool => Value(0 bytes)
0   B │ │ │─ MaxPoolMembers => Value(0 bytes)
0   B │ │ │─ MaxPools => Value(0 bytes)
8   B │ │─┬ FastUnstake

4   B │ │ │─ ErasToCheckPerBlock => Value(4 bytes)
4   B │ │ │─ CounterForQueue => Value(4 bytes)
0   B │ │ │─ Queue => Map(0 bytes, 0 keys)
0   B │ │ │─ Head => Value(0 bytes)
243 B │ │─┬ Configuration

242 B │ │ │─ ActiveConfig => Value(242 bytes)
1   B │ │ │─ PendingConfigs => Value(1 bytes)
0   B │ │ │─ BypassConsistencyCheck => Value(0 bytes)
10  K │ │─┬ ParasShared

9   K │ │ │─ ActiveValidatorKeys => Value(9,602 bytes)
1   K │ │ │─ ActiveValidatorIndices => Value(1,202 bytes)
197 B │ │ │─ AllowedRelayParents => Value(197 bytes)
4   B │ │ │─ CurrentSessionIndex => Value(4 bytes)
16  K │ │─┬ ParaInclusion

6   K │ │ │─ PendingAvailability => Map(7,140 bytes, 17 keys)
5   K │ │ │─ AvailabilityBitfields => Map(5,700 bytes, 300 keys)
4   K │ │ │─ PendingAvailabilityCommitments => Map(4,097 bytes, 17 keys)
8   K │ │─┬ ParaInherent

8   K │ │ │─ OnChainVotes => Value(8,912 bytes)
0   B │ │ │─ Included => Value(0 bytes)
2   K │ │─┬ ParaScheduler

1   K │ │ │─ ClaimQueue => Value(1,361 bytes)
1   K │ │ │─ ValidatorGroups => Value(1,302 bytes)
323 B │ │ │─ AvailabilityCores => Value(323 bytes)
4   B │ │ │─ SessionStartBlock => Value(4 bytes)
103 M │ │─┬ Paras

103 M │ │ │─ CodeByHash => Map(108,944,900 bytes, 89 keys)
20  K │ │ │─ Heads => Map(20,930 bytes, 91 keys)
2   K │ │ │─ CurrentCodeHash => Map(2,912 bytes, 91 keys)
356 B │ │ │─ CodeByHashRefs => Map(356 bytes, 89 keys)
300 B │ │ │─ PastCodeMeta => Map(300 bytes, 50 keys)
205 B │ │ │─ Parachains => Value(205 bytes)
204 B │ │ │─ MostRecentContext => Map(204 bytes, 51 keys)
96  B │ │ │─ PastCodeHash => Map(96 bytes, 3 keys)
91  B │ │ │─ ParaLifecycles => Map(91 bytes, 91 keys)
25  B │ │ │─ PastCodePruning => Value(25 bytes)
1   B │ │ │─ UpcomingUpgrades => Value(1 bytes)
1   B │ │ │─ UpgradeCooldowns => Value(1 bytes)
1   B │ │ │─ PvfActiveVoteList => Value(1 bytes)
0   B │ │ │─ UpcomingParasGenesis => Map(0 bytes, 0 keys)
0   B │ │ │─ ActionsQueue => Map(0 bytes, 0 keys)
0   B │ │ │─ UpgradeRestrictionSignal => Map(0 bytes, 0 keys)
0   B │ │ │─ UpgradeGoAheadSignal => Map(0 bytes, 0 keys)
0   B │ │ │─ FutureCodeHash => Map(0 bytes, 0 keys)
0   B │ │ │─ FutureCodeUpgrades => Map(0 bytes, 0 keys)
0   B │ │ │─ PvfActiveVoteMap => Map(0 bytes, 0 keys)
0   B │ │─┬ Initializer

0   B │ │ │─ BufferedSessionChanges => Value(0 bytes)
0   B │ │ │─ HasInitialized => Value(0 bytes)
2   K │ │─┬ Dmp

1   K │ │ │─ DownwardMessageQueueHeads => Map(1,696 bytes, 53 keys)
816 B │ │ │─ DeliveryFeeFactor => Map(816 bytes, 51 keys)
71  B │ │ │─ DownwardMessageQueues => Map(71 bytes, 71 keys)
19  K │ │─┬ Hrmp

17  K │ │ │─ HrmpChannels => Map(17,765 bytes, 209 keys)
828 B │ │ │─ HrmpChannelContents => Map(828 bytes, 3 keys)
726 B │ │ │─ HrmpOpenChannelRequests => Map(726 bytes, 22 keys)
280 B │ │ │─ HrmpWatermarks => Map(280 bytes, 70 keys)
177 B │ │ │─ HrmpOpenChannelRequestsList => Value(177 bytes)
170 B │ │ │─ HrmpEgressChannelsIndex => Map(170 bytes, 34 keys)
170 B │ │ │─ HrmpIngressChannelsIndex => Map(170 bytes, 34 keys)
70  B │ │ │─ HrmpChannelDigests => Map(70 bytes, 70 keys)
44  B │ │ │─ HrmpOpenChannelRequestCount => Map(44 bytes, 11 keys)
0   B │ │ │─ HrmpCloseChannelRequestsList => Value(0 bytes)
0   B │ │ │─ HrmpCloseChannelRequests => Map(0 bytes, 0 keys)
0   B │ │ │─ HrmpAcceptedChannelRequestCount => Map(0 bytes, 0 keys)
464 K │ │─┬ ParaSessionInfo

367 K │ │ │─ Sessions => Map(376,390 bytes, 7 keys)
65  K │ │ │─ AccountKeys => Map(67,214 bytes, 7 keys)
31  K │ │ │─ AssignmentKeysUnsafe => Value(32,002 bytes)
182 B │ │ │─ SessionExecutorParams => Map(182 bytes, 7 keys)
4   B │ │ │─ EarliestStoredSession => Value(4 bytes)
281 K │ │─┬ ParasDisputes

281 K │ │ │─ Included => Map(288,396 bytes, 72099 keys)
89  B │ │ │─ Disputes => Map(89 bytes, 1 keys)
9   B │ │ │─ BackersOnDisputes => Map(9 bytes, 1 keys)
4   B │ │ │─ LastPrunedSession => Value(4 bytes)
1   B │ │ │─ Frozen => Value(1 bytes)
0   B │ │─┬ ParasSlashing

0   B │ │ │─ ValidatorSetCounts => Map(0 bytes, 0 keys)
0   B │ │ │─ UnappliedSlashes => Map(0 bytes, 0 keys)
0   B │ │─┬ OnDemandAssignmentProvider

0   B │ │ │─ ParaIdAffinity => Map(0 bytes, 0 keys)
0   B │ │ │─ OnDemandQueue => Value(0 bytes)
0   B │ │ │─ SpotTraffic => Value(0 bytes)
935 B │ │─┬ CoretimeAssignmentProvider

935 B │ │ │─ CoreDescriptors => Map(935 bytes, 55 keys)
0   B │ │ │─ CoreSchedules => Map(0 bytes, 0 keys)
5   K │ │─┬ Registrar

5   K │ │ │─ Paras => Map(5,400 bytes, 108 keys)
12  B │ │ │─ PendingSwap => Map(12 bytes, 3 keys)
4   B │ │ │─ NextFreeParaId => Value(4 bytes)
8   K │ │─┬ Slots

8   K │ │ │─ Leases => Map(8,436 bytes, 57 keys)
4   B │ │─┬ Auctions

4   B │ │ │─ AuctionCounter => Value(4 bytes)
0   B │ │ │─ Winning => Map(0 bytes, 0 keys)
0   B │ │ │─ ReservedAmounts => Map(0 bytes, 0 keys)
0   B │ │ │─ AuctionInfo => Value(0 bytes)
3   K │ │─┬ Crowdloan

3   K │ │ │─ Funds => Map(3,264 bytes, 32 keys)
4   B │ │ │─ NextFundIndex => Value(4 bytes)
4   B │ │ │─ EndingsCount => Value(4 bytes)
0   B │ │ │─ NewRaise => Value(0 bytes)
2   K │ │─┬ XcmPallet

1020B │ │ │─ AssetTraps => Map(1,020 bytes, 255 keys)
384 B │ │ │─ VersionNotifiers => Map(384 bytes, 48 keys)
384 B │ │ │─ Queries => Map(384 bytes, 48 keys)
357 B │ │ │─ VersionNotifyTargets => Map(357 bytes, 21 keys)
160 B │ │ │─ SupportedVersion => Map(160 bytes, 40 keys)
8   B │ │ │─ QueryCounter => Value(8 bytes)
4   B │ │ │─ SafeXcmVersion => Value(4 bytes)
1   B │ │ │─ VersionDiscoveryQueue => Value(1 bytes)
0   B │ │ │─ XcmExecutionSuspended => Value(0 bytes)
0   B │ │ │─ LockedFungibles => Map(0 bytes, 0 keys)
0   B │ │ │─ RemoteLockedFungibles => Map(0 bytes, 0 keys)
0   B │ │ │─ CurrentMigration => Value(0 bytes)
1   K │ │─┬ MessageQueue

1015B │ │ │─ BookStateFor => Map(1,015 bytes, 35 keys)
540 B │ │ │─ Pages => Map(540 bytes, 6 keys)
0   B │ │ │─ ServiceHead => Value(0 bytes)
16  B │ │─┬ AssetRate

16  B │ │ │─ ConversionRateToNative => Map(16 bytes, 1 keys)
65  K │ │─┬ Beefy

32  K │ │ │─ NextAuthorities => Value(33,002 bytes)
32  K │ │ │─ Authorities => Value(33,002 bytes)
672 B │ │ │─ SetIdSession => Map(672 bytes, 168 keys)
8   B │ │ │─ ValidatorSetId => Value(8 bytes)
5   B │ │ │─ GenesisBlock => Value(5 bytes)
328 B │ │─┬ Mmr

288 B │ │ │─ Nodes => Map(288 bytes, 9 keys)
32  B │ │ │─ RootHash => Value(32 bytes)
8   B │ │ │─ NumberOfLeaves => Value(8 bytes)
88  B │ │─┬ BeefyMmrLeaf

44  B │ │ │─ BeefyNextAuthorities => Value(44 bytes)
44  B │ │ │─ BeefyAuthorities => Value(44 bytes)

Total size: 172 M kusama
438.03726s

@paritytech-cicd-pr
Copy link

The CI pipeline was cancelled due to failure one of the required jobs.
Job name: check-tracing
Logs: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6113715


#[derive(Debug, Clone, Default)]
struct Pallet {
pub name: String,
Copy link
Contributor

@kianenigma kianenigma May 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably need not be pub, or else should be documented better.


let mut total_size = 0;
match metadata {
RuntimeMetadata::V14(inner) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We at least need to support Metadata V15 as well.

Moreover, I wonder if this whole crate can now be done better with subxt, or frame-metadata crate?

Copy link
Contributor

@kianenigma kianenigma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a good effort but tbh this is mostly the code from 3 years ago being revived today, and I doubt if it is useful nowadays..

I belive a modern approach to solving this would be to extend https://paritytech.github.io/subxt-explorer/ to also show the size of storage items, if requested. cc @jsdw

Thank you for your attempt at fixing the issue though!

@kianenigma kianenigma mentioned this pull request May 7, 2024
1 task
@dastansam
Copy link
Contributor Author

It is a good effort but tbh this is mostly the code from 3 years ago being revived today, and I doubt if it is useful nowadays..

I belive a modern approach to solving this would be to extend https://paritytech.github.io/subxt-explorer/ to also show the size of storage items, if requested. cc @jsdw

Thank you for your attempt at fixing the issue though!

yeah, the issue is pretty old, it makes sense that requirements might have changed. It didn't really take much time, so I will close it. maybe open an issue in the subxt-explorer.

the only thing I am wondering is whether it would be feasible to put this on an explorer. For reading some relatively big storage entries (e.g System.Account), it takes quite a while - around 1-2 min, which is not ideal for UX.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

revive sub-du in utils/frame
3 participants