[Networking] Handling iHave overpromising part-1

[yhassanzadeh13]

This PR introduces a comprehensive set of parameters for GossipSub's message delivery scoring mechanism, providing a way to calculate the performance of each peer in a topic mesh. It enables a decay system to keep the score dynamic and relevant and includes measures to prevent any potential message replay attacks.

The parameters are explained as follows:

• Message Delivery Decay (defaultTopicMeshMessageDeliveriesDecay): Controls the decay rate of the count of actual message deliveries by a peer in a topic mesh.
• Message Delivery Cap (defaultTopicMeshMessageDeliveriesCap): Sets an upper limit on the number of actual message deliveries that contribute to the score of a peer.
• Message Delivery Threshold (defaultTopicMeshMessageDeliveryThreshold): Establishes a baseline for the number of actual message deliveries. If the actual count is below this threshold, the peer is penalized.
• Message Delivery Weight (defaultTopicMeshMessageDeliveriesWeight): Provides the weight for the penalty if a peer underperforms.
• Message Delivery Window (defaultMeshMessageDeliveriesWindow): Defines the window size, within which a message delivery counts towards the score of a peer.
• Message Delivery Activation (defaultMeshMessageDeliveriesActivation): Specifies the time we wait for a new peer to start counting its message deliveries.

The scoring mechanism operates per topic and contributes to the overall score of a peer. With the introduction of decay parameters, the score of a peer is continually updated, thus incentivizing active participation. Additionally, the mechanism is equipped to deal with replay attacks, disregarding any message replayed within the same sender's window size.

This PR plays a significant role in our ongoing efforts to mitigate the threat of iHave overpromising spam within our network. We've designed the message delivery scoring system to identify and penalize an attacker who overpromises iHave messages but consistently under-delivers. Our approach, however, doesn't immediately disconnect from a node solely due to under-performance alone. Instead, we've implemented a strategy where the under-performing node accrues sufficient penalties over time, causing a significant decrease in its overall score. As a result of this cumulative penalization, the under-performing node naturally becomes less preferred by honest nodes when they compare it to other peers with higher scores. This mechanism subtly discourages the participation of such actors, thus promoting overall network integrity and reliability, while still maintaining connections to avoid potential false positives.

[codecov-commenter]

Codecov Report

Merging #4556 (493a8f5) into master (2528190) will decrease coverage by 1.43%.
The diff coverage is 49.52%.

@@            Coverage Diff             @@
##           master    #4556      +/-   ##
==========================================
- Coverage   56.25%   54.82%   -1.43%     
==========================================
  Files         653      816     +163     
  Lines       64699    78639   +13940     
==========================================
+ Hits        36396    43114    +6718     
- Misses      25362    32102    +6740     
- Partials     2941     3423     +482     

Flag	Coverage Δ
unittests	54.82% <49.52%> (-1.43%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
cmd/scaffold.go	14.83% <0.00%> (+0.06%)	⬆️
engine/access/rpc/backend/backend_transactions.go	46.07% <0.00%> (-0.08%)	⬇️
model/flow/identifierList.go	22.36% <0.00%> (+0.84%)	⬆️
model/verification/chunkDataPackRequest.go	43.75% <0.00%> (-6.25%)	⬇️
module/metrics/herocache.go	0.00% <0.00%> (ø)
module/metrics/network.go	0.00% <0.00%> (ø)
module/metrics/noop.go	0.00% <0.00%> (ø)
storage/badger/cleaner.go	0.00% <0.00%> (ø)
utils/unittest/chain_suite.go	0.00% <0.00%> (ø)
utils/unittest/execution_state.go	31.81% <0.00%> (ø)
... and 17 more

... and 158 files with indirect coverage changes

[tarakby]

Thanks for the extensive and clear README updates!

I didn't quite get one point: If an under-performing node has its score reduced among the peers, and the peers stop getting messages from this node, how does this node get penalized? In other words, is there a protocol reward for exchanging honest messages that the under-performing node would lose?

[tarakby]

Is the TODO still valid?

[yhassanzadeh13]

Yes, this TODO means that eventually, peer scoring must be a default setting for the gossipsub rather than setting it through builder, i.e., every gossipsub instance we make at our codebase must have the peer scoring enabled, and disabling the peer scoring must not be an option, as it poses a security vulnerability.

[yhassanzadeh13]

Thanks for the extensive and clear README updates!

I didn't quite get one point: If an under-performing node has its score reduced among the peers, and the peers stop getting messages from this node, how does this node get penalized? In other words, is there a protocol reward for exchanging honest messages that the under-performing node would lose?

@tarakby, thanks for your review. With these settings, an underperforming node has a reduced score. Hence, we disincentivise the nodes to be free-rider or to spend their resources on spamming (though we have more severe spam prevention mechanisms), or deviating from ihave-iwant lazy pull protocol by just promising iHaves and never replying iwants. When a node has a reduced score, other peers will not select it as part of their local mesh. In GossipSub, a new message is disseminated in two ways: (1) eager push, where each node pushes a new message to its local mesh (2) lazy pull, where for each new message, a node picks some other nodes than its local mesh at random and advertises an ihave to them. Intuitively, lazy pull is slower than eager push. So when a node underperforms, it will be deprived of being part of many nodes local meshes. Hence, it does not benefit from receiving new messages promptly. With this PR, nodes expect each other to deliver a minimum threshold of new messages per heartbeat to be in each others' meshes. This addresses underperforming nodes, message-dropping attacks, and part of the iHave overpromising attacks, and incentives the nodes to be performant, keep pushing the new messages to their local mesh (eager push), and be responsive to iHave messages they send. In a subsequent PR, we address mitigating iHave broken promises that a malicious node intentionally keeps sending iHaves without responding the iWants. There, not only the node is penalized for underperforming due to the lack of new message delivery in a timely fashion, but also is severely penalized for persisting an iHave broken promises attack.

In other words, is there a protocol reward for exchanging honest messages that the under-performing node would lose?
There are two protocol rewards that an under-performing node will lose:

1. We give A default reward to all staked Flow nodes, and an under-performing node loses that reward partially per topic that it under-performs. If it under-performs in all topics, it eventually loses the entire reward.
2. (Not activated yet) The first message delivery reward that the node can get when it is the first node sending a new message to another node. When the node is not part of the many local meshes, it cannot make direct eager pushes, hence, it cannot benefit from this reward as much as local mesh nodes will benefit.