Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PolyBFT consensus protocol #899

Merged
merged 136 commits into from
Feb 3, 2023
Merged

PolyBFT consensus protocol #899

merged 136 commits into from
Feb 3, 2023

Conversation

Stefan-Ethernal
Copy link
Collaborator

@Stefan-Ethernal Stefan-Ethernal commented Nov 10, 2022

Description

Placeholder PR for PolyBFT consensus protocol epic.

Changes include

  • Bugfix (non-breaking change that solves an issue)
  • Hotfix (change that solves an urgent issue, and requires immediate attention)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (change that is not backwards-compatible and/or changes current functionality)

Breaking changes

Please complete this section if any breaking changes have been made, otherwise delete it

Checklist

  • I have assigned this PR to myself
  • I have added at least 1 reviewer
  • I have added the relevant labels
  • I have updated the official documentation
  • I have added sufficient documentation in code

Testing

  • I have tested this code with the official test suite
  • I have tested this code manually

Manual tests

Please complete this section if you ran manual tests for this functionality, otherwise delete it

Nemanja0x and others added 18 commits October 13, 2022 13:57
* Move PolyBFT to edge

Co-authored-by: Igor Crevar <crewce@gmail.com>
Co-authored-by: Sasa Prsic <sasa@ethernal.tech>
Co-authored-by: Stefan Negovanović <stefan@ethernal.tech>
Co-authored-by: Victor Castell <victor@victorcastell.com>
Co-authored-by: Stefan Negovanović <stefan@ethernal.tech>
Co-authored-by: Sasa Prsic <sasa@ethernal.tech>
* EVM-57 PolyBFT bridge enabled state txs

Co-authored-by: Sasa Prsic <sasa@ethernal.tech>

* EVM-57 PolyBFT bridge enabled state txs

Co-authored-by: Sasa Prsic <sasa@ethernal.tech>

* tidy.

* pr fixes

* pr fix 2

* small fix

* TransactionType should be populated in receipt

* nonce fix

* write change original tx fix

* createStateTransactionWithData change

* pr fix more

* revert ComputeHash for tx because id does not really matter

Co-authored-by: Sasa Prsic <sasa@ethernal.tech>
Back-port the e2e framework + tests from v3 to edge as a separate set of tests.
* Initial changes

* Init v3-contract submodule

* Uptime changes and linter fixes

* Use local build and test workflows (temporary)

* Test workflow expand with init git submodules

* Native transfer precompile

* CommitTransaction in blockBuilder

* adding workflow call secrets input (#799)

* Add BLS aggregated signatures verification precompile

* Change required gas for introduced precompiles

* Add console precompile

* Additional functions in bls package

* Register validator command

* Linter fixes, register subcommand for register validator command

* Adapt genesis command

* Cleanup

* Change addresses of sidechain contracts so that they don't clash with precompile addresses

* Marshal from address in transaction

* Remove CommitTransaction and use WriteTx instead

* Fix panic in register validator command

* Get rid of panics in register validator

* Increase gas limit for ChildValidatorSet initialization

* Cleanup GenesisPostHookFactory

* Fix register validator command

* Change gas limit to 100M for initialize child validator set

* Change make compile v3-contracts

* Recover CI (1st attempt)

* Recover CI (2nd attempt)

* Recover CI (3rd attempt)

* Cleanup Makefile (remove compilation of SCs when unneeded). Remove cloning submodules in lint workflow

* Recover lint

* Remove init submodules from test workflow

* Address some go security issues

* small make fix

* init root contracts command

* Remove rootchain contracts deployment from rootchain server command

* Provide new url for smart contracts

* Introduce constants for register validator and contracts root folder

* Comments

* Comments (part 2)

* Reference dev-ethernal commit from core-contracts

* Fix polybft e2e workflow

* Fix genesis cmd invocation from e2e framework

* contract_initializer fix

* Fix polybft e2e tests

* Comments (part 3)

* Populate validators balance based on provided genesis configuration.
Comments (part 4)

* Fix balance unmarshal for polybft.Validator

* Initialize native transfer contract

Co-authored-by: Goran Rojovic <goran.rojovic@ethernal.tech>
Co-authored-by: Epikichi <78609649+epikichi@users.noreply.github.com>
Co-authored-by: Igor Crevar <crewce@gmail.com>
* Add bridge flag.

* Populate bridge config.

* Fix bridge address.

* rootchain panic fixes

* changes. deploying contracts to root still does not work in test

* fix deploying contracts and running bridge. Emit command change

* Read balance from genesis

* Remove Geth reference from Edge module

* bridge test fixes

* remove test-bootnode. add WithBootnodeCount to test-cluster. Change network_test to be more relaxed

Co-authored-by: Sasa Prsic <sasa@ethernal.tech>
Co-authored-by: Igor Crevar <crewce@gmail.com>
* test-bridge fix + temporary fix

* Read balance from genesis

* state, state transaction, event tracker

* FSM tests (part 1)

* ConsensusRuntime tests

* FSM tests (part 2)

* FSM tests (part 3)

* SystemState tests

* ValidatorAccount and ValidatorSnapshot tests

* buildLogsFromReceipts test

* FSM tests (part 4)

* FSM fix remaining tests

* t.parallel

* Introduce parallel execution in tests (part 2)

* Lint fixes

* Rename test helpers file to exclude it from build

* Add test suffix to mocks

* Install solidity compiler on test workflow

* Assign execute permission to setup-ci.sh script

* polybft tests

* all tests merged

* setting epoch snapshot added

* Lint fixes

* Use constant for IstanbulExtraVanity

* Description of polybft test

Co-authored-by: Igor Crevar <crewce@gmail.com>
Co-authored-by: Nemanja0x <nemanja@ethernal.tech>
Co-authored-by: Goran Rojovic <goran.rojovic@ethernal.tech>
* Exit event storage and query functions

* ExitProof generation

* Reorganize tests

* Optimize key composition

* Extract common function for decoding state sync and exit events

* Comments fix

* Comments fix

* [Checkpointing] JSON RPC point for generating exit proofs (#844)

* JSON RPC point for generating exit proofs

* Comments fix

Co-authored-by: Stefan Negovanović <stefan@ethernal.tech>
* Add CheckpointData to Extra

* Don't store chainID, blockNumber and blockHash to the blockchain

* Set checkpoint data to the Header.Extra

* Fix and simplify FSM unit tests

* getSignHash static function, fix verify header implementation

* Increase block waiting time in bridge test

* Remove unnecessary condition

* Set checkpoint to each extra

* Sign checkpoint hash instead of aggregated hash

* Make nextValidatorsHash same as currentValidatorsHash for non-epoch ending blocks

* Set event root to the checkpoint object
* Parse receipt logs and insert exit events

* Comments fix

* Comments fix

* Comments fix

* Remove TODO

* Comments fix

* Comments fix

* Comments fix
* Change error messages for checkpoint hash calculation

* Determine checkpoint blocks

* Checkpoint submission (draft)

* Introduce checkpoint manager

* Lint fixes

* Fix tests

* Checkpoint manager checkpoint data ABI encode unit tests

* Unit test for getCurrentCheckpointID

* Unit test for submitCheckpoint

* Remove leftover code

* Comments

* Determine epoch ending blocks by comparing epoch numbers in adjacent headers

* Micro optimization

* Skip querying first block for end of epoch

* Optimization

* Simplify checkpoints offset

* Adapt comment

* Checkpoint block detection optimization (#872)

* Simplify checkpoint block detection

* Resolve conflicts and pass tests

Co-authored-by: Boris <boris@Boriss-MacBook-Pro.local>

* Simplify

* Submit pending checkpoints in unit test

* Extract parentHeader retrieval outside of the for

* Submit end of epoch blocks for pending checkpoints

* Update submitCheckpoint comment

Co-authored-by: Victor Castell <victor@polygon.technology>

* Feedback

* Separate import declarations

* Create var block for `parentExtra` and `parentHeader`

Co-authored-by: Victor Castell <victor@polygon.technology>

Co-authored-by: Goran Rojovic <goran.rojovic@ethernal.tech>
Co-authored-by: Boris <boris@Boriss-MacBook-Pro.local>
Co-authored-by: Victor Castell <victor@polygon.technology>
* Mock Host implementation in native transfer test

* Lint fix

* Fix polybft e2e tests
* Implement the logic for quorum check

Depending on the message type it will implement different kinds of quorum checks.

* Remove MaximumFaulty
* EVM 149 - syncer go ibft coordination (#843)

* consensus runtime backend metohod (not buildable)

* transport

* polybft - run cycle

* transport fix 2

* small changes

* less complex ibft consennsus wrapper.Example: https://replit.com/@crewce/Newest#main.go

* small changes

* runSequence better

* remove pbftTransportWrapper

* noone likes temp variables

* Comment fixed

* jm fixes

Co-authored-by: Igor Crevar <crewce@gmail.com>

* EVM-151 Use (or not) edge hooks support (#849)

* FSM that supports go-ibft backend interface (#865)

* consensus runtime backend metohod (not buildable)

* transport

* polybft - run cycle

* transport fix 2

* less complex ibft consennsus wrapper.Example: https://replit.com/@crewce/Newest#main.go

* Implement MessageConstructor interface.

* Tidy linter errors.

* runSequence better

* remove pbftTransportWrapper

* Add quorum size implementation.

* Remove one part of pbft.NodeID reference.

* Add some comments.

* IsValid, Build proposal on FSM

* Add block validation.

* Add stub for IsValidSender method.

* Sender validation is always true.

* fsm - Insert block

* simplify build proposal

* Tidy logs.

* IsValidSender

* pbft.Proposal removed

* Tidy errors.

* restartEpoch if epoch is nil, remove hook in fsm

* initialization fix

* Fix compare.

* quorum fixes and error propagation

* evm-151 changes + additional needed in this task

* Increase test timeout.

* bring back some part of fsm

* just execute syn test 100 times in a row

* Tidy part of errors.

* Revert e2e test.

* Remove unused code and move message creator interface implementation.

* minor fix

* Add comments.

* code organization - valdateSender + recoverAddress

* Fix compile after merge.

* small change

* restart epoch on better place. lastBuiltBlock and epoch should not be nil

* small reorg in polybft

* pr fix

* Update comments

* pr fixes no 2

* Add voting power to the validators (#880)

* Add voting power to the validator account

* Rename ValidatorAccount to ValidatorMetadata

* Set voting power to the existing test

* Comment fixes

* Rename files

* Address comments

* Remove extra space from the comment

* pr fixes

* Rename GetValidatorAccount to GetValidatorMetadata

Co-authored-by: Nemanja0x <nemanja@ethernal.tech>
Co-authored-by: Igor Crevar <crewce@gmail.com>
Co-authored-by: Stefan Negovanović <stefan@ethernal.tech>
Co-authored-by: Stefan Negovanović <93934272+Stefan-Ethernal@users.noreply.github.com>

* Accommodate go-ibft `BuildProposal` updated function (#893)

* Populate round info and change proposal hash

* Adopt currentRound from BuildProposal

* Fix logs

* Fix proposal hash calculation inside IsValidProposalHash

* Update go-ibft version

* Igor's feedback addressed

* Merge fix

* EVM-187 Handling bundles on both sync and consensus (#882)

* EVM-187 Handling bundles on both sync and consensus

* consensus_runtime tests

* UT fix

Co-authored-by: Goran Rojovic <goran.rojovic@ethernal.tech>

* Implement HasQuorum function in consensusRuntime

Co-authored-by: Nemanja Nedic <93836814+Nemanja0x@users.noreply.github.com>
Co-authored-by: Igor Crevar <crewce@gmail.com>
Co-authored-by: Saša Pršić <93726535+0xSasaPrsic@users.noreply.github.com>
Co-authored-by: Nemanja0x <nemanja@ethernal.tech>
Co-authored-by: Goran Rojovic <goran.rojovic@ethernal.tech>
@Stefan-Ethernal Stefan-Ethernal added don't merge Please don't merge this functionality temporarily feature New update to Polygon Edge labels Nov 10, 2022
@Stefan-Ethernal Stefan-Ethernal changed the title Feature/v3 parity V3 Parity Nov 10, 2022
@codecov
Copy link

codecov bot commented Nov 10, 2022

Codecov Report

Merging #899 (4af23f6) into develop (df32bb4) will increase coverage by 0.63%.
The diff coverage is 57.19%.

@@             Coverage Diff             @@
##           develop     #899      +/-   ##
===========================================
+ Coverage    54.04%   54.68%   +0.63%     
===========================================
  Files          132      176      +44     
  Lines        17724    23553    +5829     
===========================================
+ Hits          9579    12880    +3301     
- Misses        7504     9645    +2141     
- Partials       641     1028     +387     
Impacted Files Coverage Δ
archive/restore.go 65.59% <0.00%> (ø)
consensus/ibft/consensus_backend.go 5.17% <0.00%> (-0.07%) ⬇️
consensus/ibft/fork/helper.go 81.81% <0.00%> (ø)
consensus/ibft/ibft.go 1.45% <0.00%> (-0.01%) ⬇️
consensus/polybft/block_builder.go 0.00% <0.00%> (ø)
consensus/polybft/contractsapi/helper.go 0.00% <0.00%> (ø)
consensus/polybft/ibft_consensus.go 0.00% <0.00%> (ø)
consensus/polybft/state_stats.go 0.00% <0.00%> (ø)
consensus/polybft/state_sync_manager.go 70.10% <ø> (ø)
consensus/polybft/state_transaction.go 63.63% <ø> (ø)
... and 100 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@epikichi epikichi temporarily deployed to devnet-v3-parity November 16, 2022 10:54 Inactive
@epikichi epikichi temporarily deployed to devnet-v3-parity November 16, 2022 11:03 Inactive
@epikichi epikichi temporarily deployed to devnet-v3-parity November 16, 2022 11:03 Inactive
@vcastellm vcastellm temporarily deployed to devnet-ephemeral February 2, 2023 14:19 — with GitHub Actions Inactive
@vcastellm vcastellm temporarily deployed to devnet-ephemeral February 2, 2023 14:19 — with GitHub Actions Inactive
@vcastellm vcastellm temporarily deployed to devnet-ephemeral February 2, 2023 14:28 — with GitHub Actions Inactive
@vcastellm vcastellm temporarily deployed to devnet-ephemeral February 2, 2023 14:28 — with GitHub Actions Inactive
* Added --insecure flag to both `ibft` and `polybft`
* Provide insecure flag in `polybft` e2e framework
* Provide insecure flag to `ibft` e2e framework
---------

Co-authored-by: Stefan Negovanović <stefan@ethernal.tech>
@vcastellm vcastellm temporarily deployed to devnet February 2, 2023 16:55 — with GitHub Actions Inactive
@vcastellm vcastellm temporarily deployed to devnet February 2, 2023 16:55 — with GitHub Actions Inactive
@vcastellm vcastellm temporarily deployed to devnet February 2, 2023 17:56 — with GitHub Actions Inactive
@vcastellm vcastellm temporarily deployed to devnet February 3, 2023 12:15 — with GitHub Actions Inactive
@vcastellm vcastellm temporarily deployed to devnet February 3, 2023 12:15 — with GitHub Actions Inactive
@vcastellm vcastellm temporarily deployed to devnet February 3, 2023 12:15 — with GitHub Actions Inactive
goran-ethernal and others added 3 commits February 3, 2023 14:20
* Fix  validation of state transactions

* Add a comment

* Comments fix

* Add unit test

* Return error if two uptime transactions are in a block

---------

Co-authored-by: Stefan Negovanović <stefan@ethernal.tech>
* Use one struct

* Fix state sync execution

* Remove abi functions from StateSyncProof

* Comments fix

* Comment
@vcastellm vcastellm changed the title PolyBFT consensus protocol [DON'T MERGE] PolyBFT consensus protocol Feb 3, 2023
@Nemanja0x Nemanja0x self-requested a review February 3, 2023 16:09
@vcastellm vcastellm merged commit f434a9b into develop Feb 3, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Feb 3, 2023
@Stefan-Ethernal Stefan-Ethernal deleted the feature/v3-parity branch February 6, 2023 07:53
@trimixlover trimixlover restored the feature/v3-parity branch February 7, 2023 11:29
@trimixlover trimixlover deleted the feature/v3-parity branch February 7, 2023 12:16
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feature New update to Polygon Edge
Projects
None yet
Development

Successfully merging this pull request may close these issues.