Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use sync.Pool for msgId hasher #479

Merged
merged 1 commit into from
Mar 2, 2023
Merged

fix: use sync.Pool for msgId hasher #479

merged 1 commit into from
Mar 2, 2023

Conversation

richard-ramos
Copy link
Member

cc: @LNSD in case something similar could be happening in nwaku.

While executing pprof, I got the following results, where the msgIdFn accounts for 29.95% of objects in memory due to the way hashers are created when using sha256.Sum256 function. To avoid this happening, I'm using a pool so i can reuse hashers

File: waku
Build ID: 8a9ce1246202e29639f22ec01594a1f8d7eb753e
Type: inuse_objects
Time: Mar 2, 2023 at 12:52pm (UTC)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 811015, 98.84% of 820504 total
Dropped 113 nodes (cum <= 4102)
Showing top 10 nodes out of 43
      flat  flat%   sum%        cum   cum%
    245767 29.95% 29.95%     245767 29.95%  github.com/waku-org/go-waku/waku/v2/protocol/relay.msgIdFn
    196608 23.96% 53.92%     196608 23.96%  github.com/libp2p/go-cidranger/net.NetworkNumberMask.Mask
     98305 11.98% 65.90%      98305 11.98%  net.CIDRMask (inline)
     98305 11.98% 77.88%      98305 11.98%  net.IP.Mask
     76454  9.32% 87.20%      76454  9.32%  github.com/libp2p/go-cidranger.newPrefixTree (inline)
     49155  5.99% 93.19%     475137 57.91%  github.com/libp2p/go-libp2p-asn-util.newAsnStore
     32768  3.99% 97.18%      32768  3.99%  github.com/waku-org/go-discover/discover/v5wire.deriveKeys
      8192     1% 98.18%       9096  1.11%  github.com/syndtr/goleveldb/leveldb/table.(*Reader).readBlock
      5461  0.67% 98.84%       5461  0.67%  github.com/libp2p/go-mplex.NewMultiplex
         0     0% 98.84%       8806  1.07%  github.com/ethereum/go-ethereum/p2p/enode.(*DB).expireNodes
(pprof) list relay.msgIdFn
Total: 820504
ROUTINE ======================== github.com/waku-org/go-waku/waku/v2/protocol/relay.msgIdFn in /root/go-waku/waku/v2/protocol/relay/waku_relay.go
    245767     245767 (flat, cum) 29.95% of Total
         .          .     54:func msgIdFn(pmsg *pubsub_pb.Message) string {
         .          .     55:	hash := sha256.Sum256(pmsg.Data)
    245767     245767     56:	return string(hash[:])
         .          .     57:}
         .          .     58:
         .          .     59:// NewWakuRelay returns a new instance of a WakuRelay struct
         .          .     60:func NewWakuRelay(h host.Host, bcaster v2.Broadcaster, minPeersToPublish int, timesource timesource.Timesource, log *zap.Logger, opts ...pubsub.Option) *WakuRelay {
         .          .     61:	w := new(WakuRelay)
(pprof) 

@status-im-auto
Copy link

status-im-auto commented Mar 2, 2023

Jenkins Builds

Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 6147501 #1 2023-03-02 14:02:35 ~4 min linux 📦deb
✔️ 6147501 #1 2023-03-02 14:04:13 ~5 min android 📦tgz
✔️ 6147501 #1 2023-03-02 14:06:28 ~7 min ios 📦tgz

@richard-ramos richard-ramos merged commit df2ccce into master Mar 2, 2023
@richard-ramos richard-ramos deleted the fix/msgIdFn branch March 2, 2023 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants