feat(filter): enhancing protocol peer management with mutex locks

[danisharora099]

Problem

The current implementation showcased a lot of inconsistencies around peer management for protocols.
Multiple weird behaviour were observed:

• protocol not able to find new peers from available pool, while it truly existed
• peer renewal for a protocol was unsuccessful about the Peer Manager count didn't reflect that
• protocol sometimes showed a count of available peers much greater than the total actually connected peers

Solution

• Implemented mutex locks to ensure thread-safe protocol management
• Refactored peer management methods in the BaseProtocol class to enhance performance and reliability
• Improved error handling and logging in the FilterCore class
• Use async-mutex for all locks-related chores

Notes

• Resolves bug: inconsistent protocol peer management #2147
• Partially resolves bug: Filter subscriptions are not stable and missing messages #2139

Contribution checklist:

• covered by unit tests;
• covered by e2e test;
• add ! in title if breaks public API;

[github-actions]

size-limit report 📦

Path	Size	Loading time (3g)	Running time (snapdragon)	Total time
Waku node	85.46 KB (+1.47% 🔺)	1.8 s (+1.47% 🔺)	1.4 s (+37.22% 🔺)	3.1 s
Waku Simple Light Node	137.5 KB (+1.86% 🔺)	2.8 s (+1.86% 🔺)	1.6 s (+9.38% 🔺)	4.4 s
ECIES encryption	22.88 KB (-0.3% 🔽)	458 ms (-0.3% 🔽)	595 ms (-15.42% 🔽)	1.1 s
Symmetric encryption	22.37 KB (-0.12% 🔽)	448 ms (-0.12% 🔽)	391 ms (+32.42% 🔺)	838 ms
DNS discovery	72.9 KB (+0.87% 🔺)	1.5 s (+0.87% 🔺)	1.4 s (+52.9% 🔺)	2.8 s
Peer Exchange discovery	74.1 KB (+0.3% 🔺)	1.5 s (+0.3% 🔺)	1.2 s (+40.31% 🔺)	2.7 s
Local Peer Cache Discovery	68.03 KB (+0.6% 🔺)	1.4 s (+0.6% 🔺)	2.3 s (+182.33% 🔺)	3.7 s
Privacy preserving protocols	75.99 KB (+0.51% 🔺)	1.6 s (+0.51% 🔺)	1.7 s (-19.61% 🔽)	3.2 s
Waku Filter	79.75 KB (+1.5% 🔺)	1.6 s (+1.5% 🔺)	1.1 s (-38.92% 🔽)	2.7 s
Waku LightPush	77.77 KB (+1.23% 🔺)	1.6 s (+1.23% 🔺)	1.5 s (+0.72% 🔺)	3 s
History retrieval protocols	77.06 KB (+1.27% 🔺)	1.6 s (+1.27% 🔺)	967 ms (-15.28% 🔽)	2.6 s
Deterministic Message Hashing	7.39 KB (+0.15% 🔺)	148 ms (+0.15% 🔺)	171 ms (-14.07% 🔽)	319 ms

[weboko]

Ideally we should be fine with increasing this number, not decreasing.
How did you reach this number?

[danisharora099]

No particular reasoning. We can change it. 30S was a guesstimate as well

[weboko]

why is this added?

[danisharora099]

so that PeerManager is forced to find all peers until numPeersToUse which awaits the process instead

[weboko]

I see, then I think forceUseAllPeers is a bit misleading - I thought it will use all the peers

can be a follow up

[danisharora099]

Not important for now considering PeerManager is removed from LightPush: #2137 (comment)

[weboko]

added comments

[weboko]

please, address #2137 (comment) and #2137 (comment) before merging

from code perspective seems fine but after dogfooding it I see following critical issue - filter doesn't work at all as the peers are constantly getting renewed (I see it happens due to checkAndRenewPeers):

Let's disable reliabilityManager for Filter in this PR and I will follow up on it in #2158

Used version https://github.com/waku-org/js-waku/actions/runs/11249833553/job/31277485871