Recovery from unexpectedly large Eth reorg

[MaksymZavershynskyi]

This issue relates to all components of the bridge, so I have created it in this repo. It is mostly related to Ropsten, but with a very low chance it might be useful for Ethereum Mainnet.

Problem 1

Sometimes Ropsten reorgs thousands of blocks see #329 . If depth of re-org exceeds the depth of finalized_gc_threshold in EthOnNearClient then the bridge will break. Specifically, the client won't be able to switch to the new canonical chain.

We need to discuss if there is a DevX-acceptable way to address this problem.

Problem 2

If Ropsten re-orgs only exceed num_confirmations but do not exceed finalized_gc_threshold (we are transitioning this configuration parameter into connectors near/rainbow-bridge-rs#3) there are two issues:

• Some of the assets that were finalized might have been already used to provide service, and now these assets were reverted;
• The amount of assets locked on one blockchain might not correspond anymore to the number of assets minted on another blockchain, breaking invariant, and potentially leading to a broken contract;

The first issue is not addressable by engineering. The second issue however can be addressed if connectors are somehow equipped to keep history by associated minted or locked assets with the blocks at which they were locked/minted. If these blocks are reverted they should revert the assets too restoring equilibrium. For example:

• Suppose there is an ERC20 contract DAI on Ethereum;
• Suppose we have transferred in total X DAI to nearDAI;
• The amount of locked DAI is now X and amount of minted nearDAI is X, too;
• Suppose Ethereum goes through re-org where it undoes large portion of blocks resulting in only Y<X DAI being locked. As the result, it will not be possible to move X nearDAI into DAI, because there is not enough locked DAI tokens;

The connector contract should then recover in the following way:

• It observes that transfers worth of X-Y DAI were lost;
• It tracks through the history the minting transactions that correspond to these X-Y nearDAI;
• It undoes the minting of these tokens.