Skip to content

Commit

Permalink
Memberlist: Optimize conflictingTokensExist (#84)
Browse files Browse the repository at this point in the history
This PR uses a memory pool instead of allocating a map for every time we call conflictingTokensExists to allow us to gc that memory easily and prevent long lived maps. Also switch to using a map[unit32]struct{} instead of a map[uint32]bool since we use the boolean as pretty much just a contains check.
  • Loading branch information
Tyler Reid authored Dec 1, 2021
1 parent 1604ffb commit a65c443
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 9 deletions.
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
* [ENHANCEMENT] Replace go-kit/kit/log with go-kit/log. #52
* [ENHANCEMENT] Add spanlogger package. #42
* [ENHANCEMENT] Add runutil.CloseWithLogOnErr function. #58
* [ENHANCEMENT] Optimise memberlist receive path when used as a backing store for rings with a large number of members. #76 #77
* [ENHANCEMENT] Optimise memberlist receive path when used as a backing store for rings with a large number of members. #76 #77 #84
* [ENHANCEMENT] Memberlist: prepare the data to send on the write before starting counting the elapsed time for `-memberlist.packet-write-timeout`, in order to reduce chances we hit the timeout when sending a packet to other node. #89
* [BUGFIX] spanlogger: Support multiple tenant IDs. #59
* [BUGFIX] Memberlist: fixed corrupted packets when sending compound messages with more than 255 messages or messages bigger than 64KB. #85
20 changes: 12 additions & 8 deletions ring/model.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"container/heap"
"fmt"
"sort"
"sync"
"time"

"github.com/gogo/protobuf/proto"
Expand Down Expand Up @@ -302,19 +303,22 @@ func normalizeIngestersMap(inputRing *Desc) {
}
}

func conflictingTokensExist(normalizedIngesters map[string]InstanceDesc) bool {
count := 0
for _, ing := range normalizedIngesters {
count += len(ing.Tokens)
}
var tokenMapPool = sync.Pool{New: func() interface{} { return make(map[uint32]struct{}) }}

tokensMap := make(map[uint32]bool, count)
func conflictingTokensExist(normalizedIngesters map[string]InstanceDesc) bool {
tokensMap := tokenMapPool.Get().(map[uint32]struct{})
defer func() {
for k := range tokensMap {
delete(tokensMap, k)
}
tokenMapPool.Put(tokensMap)
}()
for _, ing := range normalizedIngesters {
for _, t := range ing.Tokens {
if tokensMap[t] {
if _, contains := tokensMap[t]; contains {
return true
}
tokensMap[t] = true
tokensMap[t] = struct{}{}
}
}
return false
Expand Down

0 comments on commit a65c443

Please sign in to comment.