Move Orion peer table to memory and implement environment variables for configuration

[MadelineMurray]

Overview

This bounty is to add high availability support to Orion and provide easier configurations (via environment variables) to make deployment of Orion in Kubernetes simpler.

Technical Details

The bulk of the technical work to provide high availability for Orion consists of moving Orion's peer table out of memory and storing it in the Orion database. Multiple Orion instances can then use the same database so that their peer tables are the same and do not become inconsistent.

Orion needs to provide an option for the user to delete the known nodes information from the database. We need to create a startup flag that will determine if the starting node should delete all entries from the known peers table before it starts discovering new nodes.

Support for environment variables in addition to the currently supported configuration file options will give more flexibility to how Orion can be deployed depending on infrastructure. Specifically, all CLI options and configuration file parameters should be possible to set via environment variables.

Note: It may be possible to re-use the Besu PICO CLI to reduce maintenance costs.

A good implementation will:

• Document how to configure Orion for high availability
• Have tests to validate the peer table is the same for Orion instances in the HA cluster
• Have tests to validate configuring Orion using environment variables works. For example, this is how we've done this for Besu https://github.com/hyperledger/besu/tree/master/docker/tests
• Document Orion environment variables
• Either be backwards compatible or clearly document the configuration changes needed for users upgrading from the previous Orion version
• Use an approach to environment variables in Orion that's consistent with Besu

Submission Requirements

Before beginning to implement, please share your approach on the issue so that our team can give you proactive comments.
Implement storing of Orion peer table in the Orion database
Implement use of environment variables for storing Orion CLI and configuration file options
All submitted code must be Apache 2.0 licensed and the Contributor Licence Agreement signed for such code
All new libraries and dependencies must be Apache 2.0 friendly

Evaluation Criteria

The bounty will be awarded to the best submission based on peer table and environment variable implementation, code and testing quality, compliance with Contribution Guidelines, maintenance burdens, and the Submission Requirements above, according to PegaSys. Before beginning to implement, please share your approach on the issue so that our team can give you proactive comments.

Bounty Amount: 3,000 DAI

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 3000.0 DAI (3000.0 USD @ $1.0/DAI) attached to it.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $88,771.31 more funded OSS Work available on the Gitcoin Issue Explorer

[xiaods]

hi team, i have seen the bounty's summary, quote from the comments:

This bounty is to add high availability support to Orion and provide easier configurations (via environment variables) to make deployment of Orion in Kubernetes simpler.

so we only need consider k8s environment? use helm to deploy orion?

[xiaods]

@MadelineMurray i want to discussion with team, breakdown the task to some sub-task to do it asap. any chance to share the chat channel let me to ask some question?

[timbeiko]

Hi @xiaods @mul1sh,

Sorry about that, but the bounty is reserved for @atoulme. We tried to mark it on Gitcoin but there was a bug and it couldn't be reserved for him on the platform.

We should be putting out more bounties in the next few weeks, so please keep an eye out on this repo.

Thanks!

[mul1sh]

@timbeiko no worries thanks for the heads up

[xiaods]

@timbeiko Thanks for reminder.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work has been started.

These users each claimed they can complete the work by 2 months, 1 week from now.
Please review their action plans below:

1) atoulme has been approved to start work.

• Implement a persistent version of ConcurrentNetworkNodes. Make sure all tests still pass and have no functional changes - such as no breakage of API or JSON sent over the wire.
• Implement env var flags as part of the Config class. The env vars should take over the config toml file values.

This is in progress under the branch persistent_nodes under my repository here:
https://github.com/atoulme/orion/tree/persistent_nodes

I will open a draft PR soon.

Learn more on the Gitcoin Issue Details page.

[timbeiko]

@atoulme you're good to go 😄

[gitcoinbot]

@atoulme Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• reminder (3 days)
• escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[timbeiko]

Sorry about the ping, Antoine. Snoozed for 100 days.

[atoulme]

No worries. The PR is here! Consensys/orion#332

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work for 3000.0 DAI (3000.0 USD @ $1.0/DAI) has been submitted by:

1. @atoulme

@timbeiko please take a look at the submitted work:

• PR by @atoulme

---

• Learn more on the Gitcoin Issue Details page
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $76,682.61 more funded OSS Work available on the Gitcoin Issue Explorer

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

The funding of 3000.0 DAI (3000.0 USD @ $1.0/DAI) attached to this issue has been approved & issued to @atoulme.

• Learn more on the Gitcoin Issue Details page
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $73,191.24 more funded OSS Work available on the Gitcoin Issue Explorer

[timbeiko]

@atoulme sent the funds over Gitcoin. Thank for your work on this 😄

[atoulme]

👍 thanks folks, been a pleasure!