diff --git a/cumulus/README.md b/cumulus/README.md index 59e0cff015d3..4ceb9ff09107 100644 --- a/cumulus/README.md +++ b/cumulus/README.md @@ -1,13 +1,265 @@ -Dear contributors and users, +# Cumulus ☁️ -We would like to inform you that we have recently made significant changes to our repository structure. In order to streamline our development process and foster better contributions, we have merged three separate repositories Cumulus, Substrate and Polkadot into a single new repository: [the Polkadot SDK](https://github.com/paritytech/polkadot-sdk). Go ahead and make sure to support us by giving a star ⭐️ to the new repo. +[![Doc](https://img.shields.io/badge/cumulus%20docs-master-brightgreen)](https://paritytech.github.io/cumulus/) -By consolidating our codebase, we aim to enhance collaboration and provide a more efficient platform for future development. +This repository contains both the Cumulus SDK and also specific chains implemented +on top of this SDK. -If you currently have an open pull request in any of the merged repositories, we kindly request that you resubmit your PR in the new repository. This will ensure that your contributions are considered within the updated context and enable us to review and merge them more effectively. +If you only want to run a **Polkadot Parachain Node**, check out our [container section](./docs/container.md). -We appreciate your understanding and ongoing support throughout this transition. Should you have any questions or require further assistance, please don't hesitate to [reach out to us](https://forum.polkadot.network/t/psa-parity-is-currently-working-on-merging-the-polkadot-stack-repositories-into-one-single-repository/2883). +## Cumulus SDK -Best Regards, +A set of tools for writing [Substrate](https://substrate.io/)-based +[Polkadot](https://wiki.polkadot.network/en/) +[parachains](https://wiki.polkadot.network/docs/en/learn-parachains). Refer to the included +[overview](docs/overview.md) for architectural details, and the +[Connect to a relay chain how-to guide](https://docs.substrate.io/reference/how-to-guides/parachains/connect-to-a-relay-chain/) for a +guided walk-through of using these tools. -Parity Technologies \ No newline at end of file +It's easy to write blockchains using Substrate, and the overhead of writing parachains' +distribution, p2p, database, and synchronization layers should be just as low. This project aims to +make it easy to write parachains for Polkadot by leveraging the power of Substrate. + +Cumulus clouds are shaped sort of like dots; together they form a system that is intricate, +beautiful and functional. + +### Consensus + +[`parachain-consensus`](https://github.com/paritytech/cumulus/blob/master/client/consensus/common/src/parachain_consensus.rs) is a +[consensus engine](https://docs.substrate.io/v3/advanced/consensus) for Substrate +that follows a Polkadot +[relay chain](https://wiki.polkadot.network/docs/en/learn-architecture#relay-chain). This will run +a Polkadot node internally, and dictate to the client and synchronization algorithms which chain +to follow, +[finalize](https://wiki.polkadot.network/docs/en/learn-consensus#probabilistic-vs-provable-finality), +and treat as best. + +### Collator + +A Polkadot [collator](https://wiki.polkadot.network/docs/en/learn-collator) for the parachain is +implemented by the `polkadot-parachain` binary (previously called `polkadot-collator`). + +You may run `polkadot-parachain` locally after building it or using one of the container option described [here](./docs/container.md). + +### Relay Chain Interaction +To operate a parachain node, a connection to the corresponding relay chain is necessary. This can be +achieved in one of three ways: +1. Run a full relay chain node within the parachain node (default) +2. Connect to an external relay chain node via WebSocket RPC +3. Run a light client for the relay chain + +#### In-process Relay Chain Node +If an external relay chain node is not specified (default behavior), then a full relay chain node is +spawned within the same process. + +This node has all of the typical components of a regular Polkadot node and will have to fully sync +with the relay chain to work. + +##### Example command +```bash +polkadot-parachain \ + --chain parachain-chainspec.json \ + --tmp \ + -- \ + --chain relaychain-chainspec.json +``` + +#### External Relay Chain Node +An external relay chain node is connected via WebsSocket RPC by using the `--relay-chain-rpc-urls` +command line argument. This option accepts one or more space-separated WebSocket URLs to a full relay +chain node. By default, only the first URL will be used, with the rest as a backup in case the +connection to the first node is lost. + +Parachain nodes using this feature won't have to fully sync with the relay chain to work, so in general +they will use fewer system resources. + +**Note:** At this time, any parachain nodes using this feature will still spawn a significantly cut-down +relay chain node in-process. Even though they lack the majority of normal Polkadot subsystems, they +will still need to connect directly to the relay chain network. +##### Example command +```bash +polkadot-parachain \ + --chain parachain-chainspec.json \ + --tmp \ + --relay-chain-rpc-urls \ + "ws://relaychain-rpc-endpoint:9944" \ + "ws://relaychain-rpc-endpoint-backup:9944" \ + -- \ + --chain relaychain-chainspec.json +``` + +#### Relay Chain Light Client +An internal relay chain light client provides a fast and lightweight approach for connecting to the relay chain network. +It provides relay chain notifications and facilitates runtime calls. + +To specify which chain the light client should connect to, users need to supply a relay chain chain-spec as part of the relay chain arguments. + +**Note:** At this time, any parachain nodes using this feature will still spawn a significantly cut-down +relay chain node in-process. Even though they lack the majority of normal Polkadot subsystems, they +will still need to connect directly to the relay chain network. + +##### Example command +```bash +polkadot-parachain \ + --chain parachain-chainspec.json \ + --tmp \ + --relay-chain-light-client \ + -- \ + --chain relaychain-chainspec.json +``` + +## Installation and Setup +Before building Cumulus SDK based nodes / runtimes prepare your environment by following Substrate +[installation instructions](https://docs.substrate.io/main-docs/install/). + +To launch a local network, you can use [zombienet](https://github.com/paritytech/zombienet) for +quick setup and experimentation or follow the [manual setup](#manual-setup). + +### Zombienet +We use Zombienet to spin up networks for integration tests and local networks. +Follow [these installation steps](https://github.com/paritytech/zombienet#requirements-by-provider) +to set it up on your machine. A simple network specification with two relay chain nodes and one collator is +located at [zombienet/examples/small_network.toml](zombienet/examples/small_network.toml). + + +#### Which provider should I use? +Zombienet offers multiple providers to run networks. Choose the one that best fits your needs: +- **Podman:** Choose this if you want to spin up a network quick and easy. +- **Native:** Choose this if you want to develop and deploy your changes. Requires compilation +of the binaries. +- **Kubernetes:** Choose this for advanced use-cases or running on cloud-infrastructure. + +#### How to run +To run the example network, use the following commands: + +```bash +# Podman provider +zombienet --provider podman spawn ./zombienet/examples/small_network.toml + +# Native provider, assumes polkadot and polkadot-parachains binary in $PATH +zombienet --provider native spawn ./zombienet/examples/small_network.toml +``` + +### Manual Setup +#### Launch the Relay Chain + +```bash +# Clone +git clone https://github.com/paritytech/polkadot +cd polkadot + +# Compile Polkadot with the real overseer feature +cargo build --release --bin polkadot + +# Generate a raw chain spec +./target/release/polkadot build-spec --chain rococo-local --disable-default-bootnode --raw > rococo-local-cfde.json + +# Alice +./target/release/polkadot --chain rococo-local-cfde.json --alice --tmp + +# Bob (In a separate terminal) +./target/release/polkadot --chain rococo-local-cfde.json --bob --tmp --port 30334 +``` + +#### Launch the Parachain + +```bash +# Clone +git clone https://github.com/paritytech/cumulus +cd cumulus + +# Compile +cargo build --release --bin polkadot-parachain + +# Export genesis state +./target/release/polkadot-parachain export-genesis-state > genesis-state + +# Export genesis wasm +./target/release/polkadot-parachain export-genesis-wasm > genesis-wasm + +# Collator1 +./target/release/polkadot-parachain --collator --alice --force-authoring --tmp --port 40335 --rpc-port 9946 -- --chain ../polkadot/rococo-local-cfde.json --port 30335 + +# Collator2 +./target/release/polkadot-parachain --collator --bob --force-authoring --tmp --port 40336 --rpc-port 9947 -- --chain ../polkadot/rococo-local-cfde.json --port 30336 + +# Parachain Full Node 1 +./target/release/polkadot-parachain --tmp --port 40337 --rpc-port 9948 -- --chain ../polkadot/rococo-local-cfde.json --port 30337 +``` + +#### Register the parachain + +![image](https://user-images.githubusercontent.com/2915325/99548884-1be13580-2987-11eb-9a8b-20be658d34f9.png) + + +## Asset Hub 🪙 + +This repository also contains the Asset Hub runtimes. Asset Hub is a system parachain providing an +asset store for the Polkadot ecosystem. + +### Build & Launch a Node + +To run an Asset Hub node, you will need to compile the `polkadot-parachain` binary: + +```bash +cargo build --release --locked --bin polkadot-parachain +``` + +Once the executable is built, launch the parachain node via: + +```bash +CHAIN=asset-hub-westend # or asset-hub-kusama +./target/release/polkadot-parachain --chain $CHAIN +``` + +Refer to the [setup instructions](#manual-setup) to run a local network for development. + +## Contracts 📝 + +See [the `contracts-rococo` readme](parachains/runtimes/contracts/contracts-rococo/README.md) for details. + +## Bridge-hub 📝 + +See [the `bridge-hubs` readme](parachains/runtimes/bridge-hubs/README.md) for details. + +## Rococo 👑 + +[Rococo](https://polkadot.js.org/apps/?rpc=wss://rococo-rpc.polkadot.io) is becoming a +[Community Parachain Testbed](https://polkadot.network/blog/rococo-revamp-becoming-a-community-parachain-testbed/) +for parachain teams in the Polkadot ecosystem. It supports multiple parachains with the +differentiation of long-term connections and recurring short-term connections, to see which +parachains are currently connected and how long they will be connected for +[see here](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo-rpc.polkadot.io#/parachains). + +Rococo is an elaborate style of design and the name describes the painstaking effort that has gone +into this project. + +### Build & Launch Rococo Collators + +Collators are similar to validators in the relay chain. These nodes build the blocks that will +eventually be included by the relay chain for a parachain. + +To run a Rococo collator you will need to compile the following binary: + +```bash +cargo build --release --locked --bin polkadot-parachain +``` + +Once the executable is built, launch collators for each parachain (repeat once each for chain +`tick`, `trick`, `track`): + +```bash +./target/release/polkadot-parachain --chain $CHAIN --validator +``` + +You can also build [using a container](./docs/container.md). + +### Parachains + +* [Asset Hub](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo-statemint-rpc.polkadot.io#/explorer) +* [Contracts on Rococo](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo-contracts-rpc.polkadot.io#/explorer) +* [RILT](https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Frococo.kilt.io#/explorer) + +The network uses horizontal message passing (HRMP) to enable communication between parachains and +the relay chain and, in turn, between parachains. This means that every message is sent to the relay +chain, and from the relay chain to its destination parachain. diff --git a/polkadot/README.md b/polkadot/README.md index 59e0cff015d3..dcebcf727754 100644 --- a/polkadot/README.md +++ b/polkadot/README.md @@ -1,13 +1,264 @@ -Dear contributors and users, +# Polkadot -We would like to inform you that we have recently made significant changes to our repository structure. In order to streamline our development process and foster better contributions, we have merged three separate repositories Cumulus, Substrate and Polkadot into a single new repository: [the Polkadot SDK](https://github.com/paritytech/polkadot-sdk). Go ahead and make sure to support us by giving a star ⭐️ to the new repo. +Implementation of a node in Rust based on the Substrate framework. -By consolidating our codebase, we aim to enhance collaboration and provide a more efficient platform for future development. +> **NOTE:** In 2018, we split our implementation of "Polkadot" from its development framework +> "Substrate". See the [Substrate][substrate-repo] repo for git history prior to 2018. -If you currently have an open pull request in any of the merged repositories, we kindly request that you resubmit your PR in the new repository. This will ensure that your contributions are considered within the updated context and enable us to review and merge them more effectively. +[substrate-repo]: https://github.com/paritytech/substrate -We appreciate your understanding and ongoing support throughout this transition. Should you have any questions or require further assistance, please don't hesitate to [reach out to us](https://forum.polkadot.network/t/psa-parity-is-currently-working-on-merging-the-polkadot-stack-repositories-into-one-single-repository/2883). +This repo contains runtimes for the Polkadot, Kusama, and Westend networks. The README provides +information about installing the `polkadot` binary and developing on the codebase. For more +specific guides, like how to be a validator, see the +[Polkadot Wiki](https://wiki.polkadot.network/docs/getting-started). -Best Regards, +## Installation -Parity Technologies \ No newline at end of file +If you just wish to run a Polkadot node without compiling it yourself, you may +either run the latest binary from our +[releases](https://github.com/paritytech/polkadot/releases) page, or install +Polkadot from one of our package repositories. + +Installation from the Debian repository will create a `systemd` +service that can be used to run a Polkadot node. This is disabled by default, +and can be started by running `systemctl start polkadot` on demand (use +`systemctl enable polkadot` to make it auto-start after reboot). By default, it +will run as the `polkadot` user. Command-line flags passed to the binary can +be customized by editing `/etc/default/polkadot`. This file will not be +overwritten on updating polkadot. You may also just run the node directly from +the command-line. + +### Debian-based (Debian, Ubuntu) + +Currently supports Debian 10 (Buster) and Ubuntu 20.04 (Focal), and +derivatives. Run the following commands as the `root` user. + +```bash +# Import the security@parity.io GPG key +gpg --recv-keys --keyserver hkps://keys.mailvelope.com 9D4B2B6EB8F97156D19669A9FF0812D491B96798 +gpg --export 9D4B2B6EB8F97156D19669A9FF0812D491B96798 > /usr/share/keyrings/parity.gpg +# Add the Parity repository and update the package index +echo 'deb [signed-by=/usr/share/keyrings/parity.gpg] https://releases.parity.io/deb release main' > /etc/apt/sources.list.d/parity.list +apt update +# Install the `parity-keyring` package - This will ensure the GPG key +# used by APT remains up-to-date +apt install parity-keyring +# Install polkadot +apt install polkadot + +``` + +## Building + +### Install via Cargo + +Make sure you have the support software installed from the **Build from Source** section +below this section. + +If you want to install Polkadot in your PATH, you can do so with: + +```bash +cargo install --git https://github.com/paritytech/polkadot --tag polkadot --locked +``` + +### Build from Source + +If you'd like to build from source, first install Rust. You may need to add Cargo's bin directory +to your PATH environment variable. Restarting your computer will do this for you automatically. + +```bash +curl https://sh.rustup.rs -sSf | sh +``` + +If you already have Rust installed, make sure you're using the latest version by running: + +```bash +rustup update +``` + +Once done, finish installing the support software: + +```bash +sudo apt install build-essential git clang libclang-dev pkg-config libssl-dev protobuf-compiler +``` + +Build the client by cloning this repository and running the following commands from the root +directory of the repo: + +```bash +git checkout +./scripts/init.sh +cargo build --release +``` + +**Note:** compilation is a memory intensive process. We recommend having 4 GiB of physical RAM or swap available (keep in mind that if a build hits swap it tends to be very slow). + +**Note:** if you want to move the built `polkadot` binary somewhere (e.g. into $PATH) you will also need to move `polkadot-execute-worker` and `polkadot-prepare-worker`. You can let cargo do all this for you by running: + +```sh +cargo install --path . --locked +``` + +#### Build from Source with Docker + +You can also build from source using +[Parity CI docker image](https://github.com/paritytech/scripts/tree/master/dockerfiles/ci-linux): + +```bash +git checkout +docker run --rm -it -w /shellhere/polkadot \ + -v $(pwd):/shellhere/polkadot \ + paritytech/ci-linux:production cargo build --release +sudo chown -R $(id -u):$(id -g) target/ +``` + +If you want to reproduce other steps of CI process you can use the following +[guide](https://github.com/paritytech/scripts#gitlab-ci-for-building-docker-images). + +## Networks + +This repo supports runtimes for Polkadot, Kusama, and Westend. + +### Connect to Polkadot Mainnet + +Connect to the global Polkadot Mainnet network by running: + +```bash +./target/release/polkadot --chain=polkadot +``` + +You can see your node on [telemetry] (set a custom name with `--name "my custom name"`). + +[telemetry]: https://telemetry.polkadot.io/#list/Polkadot + +### Connect to the "Kusama" Canary Network + +Connect to the global Kusama canary network by running: + +```bash +./target/release/polkadot --chain=kusama +``` + +You can see your node on [telemetry] (set a custom name with `--name "my custom name"`). + +[telemetry]: https://telemetry.polkadot.io/#list/Kusama + +### Connect to the Westend Testnet + +Connect to the global Westend testnet by running: + +```bash +./target/release/polkadot --chain=westend +``` + +You can see your node on [telemetry] (set a custom name with `--name "my custom name"`). + +[telemetry]: https://telemetry.polkadot.io/#list/Westend + +### Obtaining DOTs + +If you want to do anything on Polkadot, Kusama, or Westend, then you'll need to get an account and +some DOT, KSM, or WND tokens, respectively. See the +[claims instructions](https://claims.polkadot.network/) for Polkadot if you have DOTs to claim. For +Westend's WND tokens, see the faucet +[instructions](https://wiki.polkadot.network/docs/learn-DOT#getting-westies) on the Wiki. + +## Hacking on Polkadot + +If you'd actually like to hack on Polkadot, you can grab the source code and build it. Ensure you have +Rust and the support software installed. This script will install or update Rust and install the +required dependencies (this may take up to 30 minutes on Mac machines): + +```bash +curl https://getsubstrate.io -sSf | bash -s -- --fast +``` + +Then, grab the Polkadot source code: + +```bash +git clone https://github.com/paritytech/polkadot.git +cd polkadot +``` + +Then build the code. You will need to build in release mode (`--release`) to start a network. Only +use debug mode for development (faster compile times for development and testing). + +```bash +./scripts/init.sh # Install WebAssembly. Update Rust +cargo build # Builds all native code +``` + +You can run the tests if you like: + +```bash +cargo test --workspace --release +``` + +You can start a development chain with: + +```bash +cargo build +cargo run -- --dev +``` + +Detailed logs may be shown by running the node with the following environment variables set: + +```bash +RUST_LOG=debug RUST_BACKTRACE=1 cargo run -- --dev +``` + +### Development + +You can run a simple single-node development "network" on your machine by running: + +```bash +polkadot --dev +``` + +You can muck around by heading to and choose "Local Node" from the +Settings menu. + +### Local Two-node Testnet + +If you want to see the multi-node consensus algorithm in action locally, then you can create a +local testnet. You'll need two terminals open. In one, run: + +```bash +polkadot --chain=polkadot-local --alice -d /tmp/alice +``` + +And in the other, run: + +```bash +polkadot --chain=polkadot-local --bob -d /tmp/bob --port 30334 --bootnodes '/ip4/127.0.0.1/tcp/30333/p2p/ALICE_BOOTNODE_ID_HERE' +``` + +Ensure you replace `ALICE_BOOTNODE_ID_HERE` with the node ID from the output of the first terminal. + +### Monitoring + +[Setup Prometheus and Grafana](https://wiki.polkadot.network/docs/maintain-guides-how-to-monitor-your-node). + +Once you set this up you can take a look at the [Polkadot Grafana dashboards](grafana/README.md) that we currently maintain. + +### Using Docker + +[Using Docker](doc/docker.md) + +### Shell Completion + +[Shell Completion](doc/shell-completion.md) + +## Contributing + +### Contributing Guidelines + +[Contribution Guidelines](CONTRIBUTING.md) + +### Contributor Code of Conduct + +[Code of Conduct](CODE_OF_CONDUCT.md) + +## License + +Polkadot is [GPL 3.0 licensed](LICENSE). diff --git a/substrate/README.md b/substrate/README.md index 59e0cff015d3..b6b5d0ccc9c4 100644 --- a/substrate/README.md +++ b/substrate/README.md @@ -1,13 +1,35 @@ -Dear contributors and users, +# Substrate · [![GitHub license](https://img.shields.io/badge/license-GPL3%2FApache2-blue)](#LICENSE) [![GitLab Status](https://gitlab.parity.io/parity/substrate/badges/master/pipeline.svg)](https://gitlab.parity.io/parity/substrate/pipelines) [![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](docs/CONTRIBUTING.md) [![Stack Exchange](https://img.shields.io/badge/Substrate-Community%20&%20Support-24CC85?logo=stackexchange)](https://substrate.stackexchange.com/) +

+ +

-We would like to inform you that we have recently made significant changes to our repository structure. In order to streamline our development process and foster better contributions, we have merged three separate repositories Cumulus, Substrate and Polkadot into a single new repository: [the Polkadot SDK](https://github.com/paritytech/polkadot-sdk). Go ahead and make sure to support us by giving a star ⭐️ to the new repo. +Substrate is a next-generation framework for blockchain innovation 🚀. -By consolidating our codebase, we aim to enhance collaboration and provide a more efficient platform for future development. +## Getting Started -If you currently have an open pull request in any of the merged repositories, we kindly request that you resubmit your PR in the new repository. This will ensure that your contributions are considered within the updated context and enable us to review and merge them more effectively. +Head to [docs.substrate.io](https://docs.substrate.io) and follow the [installation](https://docs.substrate.io/install/) instructions. +Then try out one of the [tutorials](https://docs.substrate.io/tutorials/). +Refer to the [Docker instructions](./docker/README.md) to quickly run Substrate, Substrate Node Template, Subkey, or to build a chain spec. -We appreciate your understanding and ongoing support throughout this transition. Should you have any questions or require further assistance, please don't hesitate to [reach out to us](https://forum.polkadot.network/t/psa-parity-is-currently-working-on-merging-the-polkadot-stack-repositories-into-one-single-repository/2883). +## Community & Support -Best Regards, +Join the highly active and supportive community on the [Substrate Stack Exchange](https://substrate.stackexchange.com/) to ask questions about use and problems you run into using this software. +Please do report bugs and [issues here](https://github.com/paritytech/substrate/issues) for anything you suspect requires action in the source. -Parity Technologies \ No newline at end of file +## Contributions & Code of Conduct + +Please follow the contributions guidelines as outlined in [`docs/CONTRIBUTING.md`](docs/CONTRIBUTING.md). +In all communications and contributions, this project follows the [Contributor Covenant Code of Conduct](docs/CODE_OF_CONDUCT.md). + +## Security + +The security policy and procedures can be found in [`docs/SECURITY.md`](docs/SECURITY.md). + +## License + +- Substrate Primitives (`sp-*`), Frame (`frame-*`) and the pallets (`pallets-*`), binaries (`/bin`) and all other utilities are licensed under [Apache 2.0](LICENSE-APACHE2). +- Substrate Client (`/client/*` / `sc-*`) is licensed under [GPL v3.0 with a classpath linking exception](LICENSE-GPL3). + +The reason for the split-licensing is to ensure that for the vast majority of teams using Substrate to create feature-chains, then all changes can be made entirely in Apache2-licensed code, allowing teams full freedom over what and how they release and giving licensing clarity to commercial teams. + +In the interests of the community, we require any deeper improvements made to Substrate's core logic (e.g. Substrate's internal consensus, crypto or database code) to be contributed back so everyone can benefit.