Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(x/staking): Migrate UnbondingDelegation to collections #17270

Merged
merged 51 commits into from
Aug 28, 2023
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
498e4d1
migrate DelegationKey to collections
likhita-809 Aug 3, 2023
4baf88e
remove GetUBDsKey
likhita-809 Aug 3, 2023
0cd5ee6
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 3, 2023
7b4a54f
add changelog
likhita-809 Aug 3, 2023
0781b59
iterator changes
likhita-809 Aug 3, 2023
6105704
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 3, 2023
426b532
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 4, 2023
a339485
fix all tests
likhita-809 Aug 4, 2023
a33bfa5
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 4, 2023
c5fd6e1
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 7, 2023
f59bc59
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 7, 2023
d8e1230
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 8, 2023
f2dc4f5
remove store usage from ubd set, get, remove funcs
likhita-809 Aug 8, 2023
d7358f6
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 8, 2023
852a59c
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 9, 2023
6fc003c
wip: try fixing keeper tests
likhita-809 Aug 9, 2023
9d7fce2
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 11, 2023
4bd48ca
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 16, 2023
b92f8a8
address review comments and fix tests
likhita-809 Aug 16, 2023
a2b3f59
fix tests
likhita-809 Aug 16, 2023
fd0a9f8
fix few tests
likhita-809 Aug 16, 2023
751dcaf
fix tests
likhita-809 Aug 18, 2023
4258997
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 18, 2023
d9b5cb5
fix usage of length prefixed addr key
likhita-809 Aug 18, 2023
d27e4da
fix integration tests
likhita-809 Aug 18, 2023
9205b1b
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 18, 2023
0e17234
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 18, 2023
5602c78
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 21, 2023
34b9aad
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 21, 2023
1726250
add diff test for unbonding delegation
likhita-809 Aug 21, 2023
b8b4f31
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 21, 2023
9854016
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 22, 2023
ad6b1d0
update changelog
likhita-809 Aug 22, 2023
ffd1912
address nits
likhita-809 Aug 22, 2023
762e742
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 22, 2023
8bd01ce
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 23, 2023
1f9a22f
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 23, 2023
2500c7c
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 23, 2023
ef9178e
wip: use UnbondingDelegations type instead of bytes
likhita-809 Aug 23, 2023
9e6414c
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 23, 2023
d3dbaba
fix keeper test
likhita-809 Aug 23, 2023
bedc479
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 24, 2023
68de5c5
address review comments
likhita-809 Aug 24, 2023
64a41da
fix simapp test
likhita-809 Aug 24, 2023
3b570f8
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 24, 2023
9b9cc74
remove unused error
likhita-809 Aug 25, 2023
9f6d759
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 25, 2023
98e063d
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 28, 2023
08ca907
nit: add godoc for key splits
likhita-809 Aug 28, 2023
ccc4a1b
spell correct
likhita-809 Aug 28, 2023
f9e0848
Merge branch 'main' of https://github.com/cosmos/cosmos-sdk into likh…
likhita-809 Aug 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ Ref: https://keepachangelog.com/en/1.0.0/

### API Breaking Changes

* (x/staking) [#17270](https://github.com/cosmos/cosmos-sdk/pull/17270) Use collections for `UnbondingDelegation`:
* remove from `types`: `GetUBDsKey`
* (types) `module.BeginBlockAppModule` has been replaced by Core API `appmodule.HasBeginBlocker`.
* (types) `module.EndBlockAppModule` has been replaced by Core API `appmodule.HasEndBlocker` or `module.HasABCIEndBlock` when needing validator updates.
* (types) [#17358](https://github.com/cosmos/cosmos-sdk/pull/17358) Remove deprecated `sdk.Handler`, use `baseapp.MsgServiceHandler` instead.
Expand Down
32 changes: 21 additions & 11 deletions simapp/export.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@ package simapp

import (
"encoding/json"
"errors"
"fmt"
"log"

cmtproto "github.com/cometbft/cometbft/proto/tendermint/types"

"cosmossdk.io/collections"
storetypes "cosmossdk.io/store/types"

servertypes "github.com/cosmos/cosmos-sdk/server/types"
Expand Down Expand Up @@ -191,17 +193,25 @@ func (app *SimApp) prepForZeroHeightGenesis(ctx sdk.Context, jailAllowedAddrs []
}

// iterate through unbonding delegations, reset creation height
err = app.StakingKeeper.IterateUnbondingDelegations(ctx, func(_ int64, ubd stakingtypes.UnbondingDelegation) (stop bool) {
for i := range ubd.Entries {
ubd.Entries[i].CreationHeight = 0
}
err = app.StakingKeeper.SetUnbondingDelegation(ctx, ubd)
if err != nil {
panic(err)
}
return false
})
if err != nil {
err = app.StakingKeeper.UnbondingDelegations.Walk(
ctx,
nil,
func(key collections.Pair[sdk.AccAddress, sdk.ValAddress], value []byte) (stop bool, err error) {
ubd, err := stakingtypes.UnmarshalUBD(app.appCodec, value)
if err != nil {
return true, err
}
likhita-809 marked this conversation as resolved.
Show resolved Hide resolved
for i := range ubd.Entries {
ubd.Entries[i].CreationHeight = 0
}
err = app.StakingKeeper.SetUnbondingDelegation(ctx, ubd)
if err != nil {
panic(err)
}
likhita-809 marked this conversation as resolved.
Show resolved Hide resolved
return false, err
},
)
if err != nil && !errors.Is(err, collections.ErrInvalidIterator) {
likhita-809 marked this conversation as resolved.
Show resolved Hide resolved
panic(err)
}
// Iterate through validators by power descending, reset bond heights, and
Expand Down
6 changes: 3 additions & 3 deletions tests/integration/staking/keeper/deterministic_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -471,7 +471,7 @@ func TestGRPCValidatorUnbondingDelegations(t *testing.T) {
ValidatorAddr: validator.OperatorAddress,
}

testdata.DeterministicIterations(f.ctx, t, req, f.queryClient.ValidatorUnbondingDelegations, 3719, false)
testdata.DeterministicIterations(f.ctx, t, req, f.queryClient.ValidatorUnbondingDelegations, 2735, false)
likhita-809 marked this conversation as resolved.
Show resolved Hide resolved
}

func TestGRPCDelegation(t *testing.T) {
Expand Down Expand Up @@ -543,7 +543,7 @@ func TestGRPCUnbondingDelegation(t *testing.T) {
DelegatorAddr: delegator1,
}

testdata.DeterministicIterations(f.ctx, t, req, f.queryClient.UnbondingDelegation, 1621, false)
testdata.DeterministicIterations(f.ctx, t, req, f.queryClient.UnbondingDelegation, 1618, false)
}

func TestGRPCDelegatorDelegations(t *testing.T) {
Expand Down Expand Up @@ -654,7 +654,7 @@ func TestGRPCDelegatorUnbondingDelegations(t *testing.T) {
DelegatorAddr: delegator1,
}

testdata.DeterministicIterations(f.ctx, t, req, f.queryClient.DelegatorUnbondingDelegations, 1302, false)
testdata.DeterministicIterations(f.ctx, t, req, f.queryClient.DelegatorUnbondingDelegations, 1296, false)
}

func TestGRPCHistoricalInfo(t *testing.T) {
Expand Down
134 changes: 49 additions & 85 deletions x/staking/keeper/delegation.go
Original file line number Diff line number Diff line change
Expand Up @@ -130,34 +130,36 @@ func (k Keeper) RemoveDelegation(ctx context.Context, delegation types.Delegatio
func (k Keeper) GetUnbondingDelegations(ctx context.Context, delegator sdk.AccAddress, maxRetrieve uint16) (unbondingDelegations []types.UnbondingDelegation, err error) {
unbondingDelegations = make([]types.UnbondingDelegation, maxRetrieve)

store := k.storeService.OpenKVStore(ctx)
delegatorPrefixKey := types.GetUBDsKey(delegator)

iterator, err := store.Iterator(delegatorPrefixKey, storetypes.PrefixEndBytes(delegatorPrefixKey))
if err != nil {
return unbondingDelegations, err
}
defer iterator.Close()

i := 0
for ; iterator.Valid() && i < int(maxRetrieve); iterator.Next() {
unbondingDelegation, err := types.UnmarshalUBD(k.cdc, iterator.Value())
if err != nil {
return unbondingDelegations, err
}
unbondingDelegations[i] = unbondingDelegation
i++
rng := collections.NewPrefixUntilPairRange[sdk.AccAddress, sdk.ValAddress](delegator)
err = k.UnbondingDelegations.Walk(
ctx,
rng,
func(key collections.Pair[sdk.AccAddress, sdk.ValAddress], value []byte) (stop bool, err error) {
unbondingDelegation, err := types.UnmarshalUBD(k.cdc, value)
if err != nil {
return true, err
}
unbondingDelegations = append(unbondingDelegations, unbondingDelegation)
i++

if i >= int(maxRetrieve) {
return true, nil
}
return false, nil
},
)
if err != nil && !errors.Is(err, collections.ErrInvalidIterator) {
return nil, err
}

return unbondingDelegations[:i], nil // trim if the array length < maxRetrieve
}

// GetUnbondingDelegation returns a unbonding delegation.
func (k Keeper) GetUnbondingDelegation(ctx context.Context, delAddr sdk.AccAddress, valAddr sdk.ValAddress) (ubd types.UnbondingDelegation, err error) {
store := k.storeService.OpenKVStore(ctx)
key := types.GetUBDKey(delAddr, valAddr)
value, err := store.Get(key)
if err != nil {
value, err := k.UnbondingDelegations.Get(ctx, collections.Join(delAddr, valAddr))
if err != nil && !errors.Is(err, collections.ErrNotFound) {
return ubd, err
}

Expand Down Expand Up @@ -195,63 +197,28 @@ func (k Keeper) GetUnbondingDelegationsFromValidator(ctx context.Context, valAdd
return ubds, nil
}

// IterateUnbondingDelegations iterates through all of the unbonding delegations.
func (k Keeper) IterateUnbondingDelegations(ctx context.Context, fn func(index int64, ubd types.UnbondingDelegation) (stop bool)) error {
store := k.storeService.OpenKVStore(ctx)
prefix := types.UnbondingDelegationKey
iterator, err := store.Iterator(prefix, storetypes.PrefixEndBytes(prefix))
if err != nil {
return err
}
defer iterator.Close()

for i := int64(0); iterator.Valid(); iterator.Next() {
ubd, err := types.UnmarshalUBD(k.cdc, iterator.Value())
if err != nil {
return err
}
if stop := fn(i, ubd); stop {
break
}
i++
}

return nil
}

// GetDelegatorUnbonding returns the total amount a delegator has unbonding.
func (k Keeper) GetDelegatorUnbonding(ctx context.Context, delegator sdk.AccAddress) (math.Int, error) {
unbonding := math.ZeroInt()
err := k.IterateDelegatorUnbondingDelegations(ctx, delegator, func(ubd types.UnbondingDelegation) bool {
for _, entry := range ubd.Entries {
unbonding = unbonding.Add(entry.Balance)
}
return false
})
return unbonding, err
}

// IterateDelegatorUnbondingDelegations iterates through a delegator's unbonding delegations.
func (k Keeper) IterateDelegatorUnbondingDelegations(ctx context.Context, delegator sdk.AccAddress, cb func(ubd types.UnbondingDelegation) (stop bool)) error {
store := k.storeService.OpenKVStore(ctx)
prefix := types.GetUBDsKey(delegator)
iterator, err := store.Iterator(prefix, storetypes.PrefixEndBytes(prefix))
if err != nil {
return err
}
defer iterator.Close()

for ; iterator.Valid(); iterator.Next() {
ubd, err := types.UnmarshalUBD(k.cdc, iterator.Value())
if err != nil {
return err
}
if cb(ubd) {
break
}
rng := collections.NewPrefixUntilPairRange[sdk.AccAddress, sdk.ValAddress](delegator)
err := k.UnbondingDelegations.Walk(
ctx,
rng,
func(key collections.Pair[sdk.AccAddress, sdk.ValAddress], value []byte) (stop bool, err error) {
ubd, err := types.UnmarshalUBD(k.cdc, value)
if err != nil {
return true, err
}
for _, entry := range ubd.Entries {
unbonding = unbonding.Add(entry.Balance)
}
return false, nil
},
)
if err != nil && !errors.Is(err, collections.ErrInvalidIterator) {
likhita-809 marked this conversation as resolved.
Show resolved Hide resolved
return unbonding, err
}

return nil
return unbonding, err
}

// GetDelegatorBonded returs the total amount a delegator has bonded.
Expand Down Expand Up @@ -328,19 +295,18 @@ func (k Keeper) HasMaxUnbondingDelegationEntries(ctx context.Context, delegatorA

// SetUnbondingDelegation sets the unbonding delegation and associated index.
func (k Keeper) SetUnbondingDelegation(ctx context.Context, ubd types.UnbondingDelegation) error {
store := k.storeService.OpenKVStore(ctx)
bz := types.MustMarshalUBD(k.cdc, ubd)

delAddr, err := k.authKeeper.AddressCodec().StringToBytes(ubd.DelegatorAddress)
if err != nil {
return err
}

store := k.storeService.OpenKVStore(ctx)
bz := types.MustMarshalUBD(k.cdc, ubd)
valAddr, err := k.validatorAddressCodec.StringToBytes(ubd.ValidatorAddress)
if err != nil {
return err
}
key := types.GetUBDKey(delAddr, valAddr)
err = store.Set(key, bz)
err = k.UnbondingDelegations.Set(ctx, collections.Join(sdk.AccAddress(delAddr), sdk.ValAddress(valAddr)), bz)
if err != nil {
return err
}
Expand All @@ -350,23 +316,21 @@ func (k Keeper) SetUnbondingDelegation(ctx context.Context, ubd types.UnbondingD

// RemoveUnbondingDelegation removes the unbonding delegation object and associated index.
func (k Keeper) RemoveUnbondingDelegation(ctx context.Context, ubd types.UnbondingDelegation) error {
delegatorAddress, err := k.authKeeper.AddressCodec().StringToBytes(ubd.DelegatorAddress)
store := k.storeService.OpenKVStore(ctx)
delAddr, err := k.authKeeper.AddressCodec().StringToBytes(ubd.DelegatorAddress)
if err != nil {
return err
}

store := k.storeService.OpenKVStore(ctx)
addr, err := k.validatorAddressCodec.StringToBytes(ubd.ValidatorAddress)
valAddr, err := k.validatorAddressCodec.StringToBytes(ubd.ValidatorAddress)
if err != nil {
return err
}
key := types.GetUBDKey(delegatorAddress, addr)
err = store.Delete(key)
err = k.UnbondingDelegations.Remove(ctx, collections.Join(sdk.AccAddress(delAddr), sdk.ValAddress(valAddr)))
if err != nil {
return err
}

return store.Delete(types.GetUBDByValIndexKey(delegatorAddress, addr))
return store.Delete(types.GetUBDByValIndexKey(delAddr, valAddr))
}

// SetUnbondingDelegationEntry adds an entry to the unbonding delegation at
Expand Down
20 changes: 15 additions & 5 deletions x/staking/keeper/genesis.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@ package keeper

import (
"context"
"errors"
"fmt"

abci "github.com/cometbft/cometbft/abci/types"

"cosmossdk.io/collections"
"cosmossdk.io/math"

sdk "github.com/cosmos/cosmos-sdk/types"
Expand Down Expand Up @@ -214,11 +216,19 @@ func (k Keeper) InitGenesis(ctx context.Context, data *types.GenesisState) (res
func (k Keeper) ExportGenesis(ctx sdk.Context) *types.GenesisState {
var unbondingDelegations []types.UnbondingDelegation

err := k.IterateUnbondingDelegations(ctx, func(_ int64, ubd types.UnbondingDelegation) (stop bool) {
unbondingDelegations = append(unbondingDelegations, ubd)
return false
})
if err != nil {
err := k.UnbondingDelegations.Walk(
ctx,
nil,
func(key collections.Pair[sdk.AccAddress, sdk.ValAddress], value []byte) (stop bool, err error) {
unbondingDelegation, err := types.UnmarshalUBD(k.cdc, value)
if err != nil {
return true, err
}
unbondingDelegations = append(unbondingDelegations, unbondingDelegation)
return false, nil
},
)
if err != nil && !errors.Is(err, collections.ErrInvalidIterator) {
likhita-809 marked this conversation as resolved.
Show resolved Hide resolved
panic(err)
}

Expand Down
24 changes: 14 additions & 10 deletions x/staking/keeper/grpc_query.go
Original file line number Diff line number Diff line change
Expand Up @@ -358,16 +358,20 @@ func (k Querier) DelegatorUnbondingDelegations(ctx context.Context, req *types.Q
return nil, err
}

store := runtime.KVStoreAdapter(k.storeService.OpenKVStore(ctx))
unbStore := prefix.NewStore(store, types.GetUBDsKey(delAddr))
pageRes, err := query.Paginate(unbStore, req.Pagination, func(key, value []byte) error {
unbond, err := types.UnmarshalUBD(k.cdc, value)
if err != nil {
return err
}
unbondingDelegations = append(unbondingDelegations, unbond)
return nil
})
_, pageRes, err := query.CollectionPaginate(
ctx,
k.UnbondingDelegations,
req.Pagination,
func(key collections.Pair[sdk.AccAddress, sdk.ValAddress], value []byte) (unbond types.UnbondingDelegation, err error) {
unbond, err = types.UnmarshalUBD(k.cdc, value)
if err != nil {
return unbond, err
}
unbondingDelegations = append(unbondingDelegations, unbond)
return unbond, nil
},
query.WithCollectionPaginationPairPrefix[sdk.AccAddress, sdk.ValAddress](delAddr),
)
if err != nil {
return nil, status.Error(codes.Internal, err.Error())
}
Expand Down
24 changes: 17 additions & 7 deletions x/staking/keeper/invariants.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,10 @@ package keeper

import (
"bytes"
"errors"
"fmt"

"cosmossdk.io/collections"
"cosmossdk.io/math"

sdk "github.com/cosmos/cosmos-sdk/types"
Expand Down Expand Up @@ -72,13 +74,21 @@ func ModuleAccountInvariants(k *Keeper) sdk.Invariant {
panic(err)
}

err = k.IterateUnbondingDelegations(ctx, func(_ int64, ubd types.UnbondingDelegation) bool {
for _, entry := range ubd.Entries {
notBonded = notBonded.Add(entry.Balance)
}
return false
})
if err != nil {
err = k.UnbondingDelegations.Walk(
ctx,
nil,
func(key collections.Pair[sdk.AccAddress, sdk.ValAddress], value []byte) (stop bool, err error) {
ubd, err := types.UnmarshalUBD(k.cdc, value)
if err != nil {
return true, err
}
for _, entry := range ubd.Entries {
notBonded = notBonded.Add(entry.Balance)
}
return false, nil
},
)
if err != nil && !errors.Is(err, collections.ErrInvalidIterator) {
likhita-809 marked this conversation as resolved.
Show resolved Hide resolved
panic(err)
}

Expand Down
Loading
Loading