New reference doc for Custom RPC V2 (#4654)

Thanks for @xlc for the original seed info, I've just fixed it up a bit
and added example links.

I've moved the comparison between eth-rpc-api and frontier outside, as
it is opinionation. I think the content there was good but should live
in the README of the corresponding repos. No strong opinion, happy
either way.

---------

Co-authored-by: Bryan Chen <xlchen1291@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: command-bot <>

docs/sdk/src/reference_docs/custom_runtime_api_rpc.rs

@@ -0,0 +1,77 @@
+//! # Custom RPC do's and don'ts
+//!
+//! **TLDR:** don't create new custom RPCs. Instead, rely on custom Runtime APIs, combined with
+//! `state_call`
+//!
+//! ## Background
+//!
+//! Polkadot-SDK offers the ability to query and subscribe storages directly. However what it does
+//! not have is [view functions](https://github.com/paritytech/polkadot-sdk/issues/216). This is an
+//! essential feature to avoid duplicated logic between runtime and the client SDK. Custom RPC was
+//! used as a solution. It allow the RPC node to expose new RPCs that clients can be used to query
+//! computed properties.
+//!
+//! ## Problems with Custom RPC
+//!
+//! Unfortunately, custom RPC comes with many problems. To list a few:
+//!
+//! - It is offchain logic executed by the RPC node and therefore the client has to trust the RPC
+//!   node.
+//! - To upgrade or add a new RPC logic, the RPC node has to be upgraded. This can cause significant
+//!   trouble when the RPC infrastructure is decentralized as we will need to coordinate multiple
+//!   parties to upgrade the RPC nodes.
+//! - A lot of boilerplate code are required to add custom RPC.
+//! - It prevents the dApp to use a light client or alternative client.
+//! - It makes ecosystem tooling integration much more complicated. For example, the dApp will not
+//!   be able to use [Chopsticks](https://github.com/AcalaNetwork/chopsticks) for testing as
+//!   Chopsticks will not have the custom RPC implementation.
+//! - Poorly implemented custom RPC can be a DoS vector.
+//!
+//! Hence, we should avoid custom RPC.
+//!
+//! ## Alternatives
+//!
+//! Generally, [`sc_rpc::state::StateBackend::call`] aka. `state_call` should be used instead of
+//! custom RPC.
+//!
+//! Usually, each custom RPC comes with a corresponding runtime API which implements the business
+//! logic. So instead of invoke the custom RPC, we can use `state_call` to invoke the runtime API
+//! directly. This is a trivial change on the dApp and no change on the runtime side. We may remove
+//! the custom RPC from the node side if wanted.
+//!
+//! There are some other cases that a simple runtime API is not enough. For example, implementation
+//! of Ethereum RPC requires an additional offchain database to index transactions. In this
+//! particular case, we can have the RPC implemented on another client.
+//!
+//! For example, the Acala EVM+ RPC are implemented by
+//! [eth-rpc-adapter](https://github.com/AcalaNetwork/bodhi.js/tree/master/packages/eth-rpc-adapter).
+//! Alternatively, the [Frontier](https://github.com/polkadot-evm/frontier) project  also provided
+//! Ethereum RPC compatibility directly in the node-side software.
+//!
+//! ## Create a new Runtime API
+//!
+//! For example, let's take a look a the process through which the account nonce can be queried
+//! through an RPC. First, a new runtime-api needs to be declared:
+#![doc = docify::embed!("../../substrate/frame/system/rpc/runtime-api/src/lib.rs", AccountNonceApi)]
+//!
+//! This API is implemented at the runtime level, always inside [`sp_api::impl_runtime_apis!`].
+//!
+//! As noted, this is already enough to make this API usable via `state_call`.
+//!
+//! ## Create a new custom RPC (Legacy)
+//!
+//! Should you wish to implement the legacy approach of exposing this runtime-api as a custom
+//! RPC-api, then a custom RPC server has to be defined.
+#![doc = docify::embed!("../../substrate/utils/frame/rpc/system/src/lib.rs", SystemApi)]
+//!
+//! ## Add a new RPC to the node (Legacy)
+//!
+//! Finally, this custom RPC needs to be integrated into the node side. This is usually done in a
+//! `rpc.rs` in a typical template, as follows:
+#![doc = docify::embed!("../../templates/minimal/node/src/rpc.rs", create_full)]
+//!
+//! ## Future
+//!
+//! - [XCQ](https://forum.polkadot.network/t/cross-consensus-query-language-xcq/7583) will be a good
+//! solution for most of the query needs.
+//! - [New JSON-RPC Specification](https://github.com/paritytech/json-rpc-interface-spec)

docs/sdk/src/reference_docs/mod.rs

@@ -108,3 +108,6 @@ pub mod frame_pallet_coupling;
 
 /// Learn about the Polkadot Umbrella crate that re-exports all other crates.
 pub mod umbrella_crate;
+
+/// Learn about how to create custom RPC endpoints and runtime APIs.
+pub mod custom_runtime_api_rpc;

substrate/frame/system/rpc/runtime-api/Cargo.toml

@@ -18,6 +18,7 @@ targets = ["x86_64-unknown-linux-gnu"]
 [dependencies]
 codec = { package = "parity-scale-codec", version = "3.6.12", default-features = false }
 sp-api = { path = "../../../../primitives/api", default-features = false }
+docify = "0.2.0"
 
 [features]
 default = ["std"]

substrate/frame/system/rpc/runtime-api/src/lib.rs

@@ -23,6 +23,7 @@
 
 #![cfg_attr(not(feature = "std"), no_std)]
 
+#[docify::export(AccountNonceApi)]
 sp_api::decl_runtime_apis! {
 	/// The API to query account nonce.
 	pub trait AccountNonceApi<AccountId, Nonce> where

substrate/utils/frame/rpc/system/Cargo.toml

@@ -16,9 +16,14 @@ workspace = true
 targets = ["x86_64-unknown-linux-gnu"]
 
 [dependencies]
-codec = { package = "parity-scale-codec", version = "3.6.12" }
-jsonrpsee = { version = "0.22.5", features = ["client-core", "macros", "server-core"] }
 futures = "0.3.30"
+codec = { package = "parity-scale-codec", version = "3.6.12" }
+docify = "0.2.0"
+jsonrpsee = { version = "0.22.5", features = [
+	"client-core",
+	"macros",
+	"server-core",
+] }
 log = { workspace = true, default-features = true }
 frame-system-rpc-runtime-api = { path = "../../../../frame/system/rpc/runtime-api" }
 sc-rpc-api = { path = "../../../../client/rpc-api" }

substrate/utils/frame/rpc/system/src/lib.rs

@@ -37,6 +37,7 @@ use sp_runtime::{legacy, traits};
 pub use frame_system_rpc_runtime_api::AccountNonceApi;
 
 /// System RPC methods.
+#[docify::export]
 #[rpc(client, server)]
 pub trait SystemApi<BlockHash, AccountId, Nonce> {
 	/// Returns the next valid index (aka nonce) for given account.

templates/minimal/node/Cargo.toml

@@ -14,6 +14,7 @@ build = "build.rs"
 targets = ["x86_64-unknown-linux-gnu"]
 
 [dependencies]
+docify = "0.2.0"
 clap = { version = "4.5.3", features = ["derive"] }
 futures = { version = "0.3.30", features = ["thread-pool"] }
 futures-timer = "3.0.1"

templates/minimal/node/src/rpc.rs

@@ -27,7 +27,6 @@ use runtime::interface::{AccountId, Nonce, OpaqueBlock};
 use sc_transaction_pool_api::TransactionPool;
 use sp_blockchain::{Error as BlockChainError, HeaderBackend, HeaderMetadata};
 use std::sync::Arc;
-use substrate_frame_rpc_system::{System, SystemApiServer};
 
 pub use sc_rpc_api::DenyUnsafe;
 
@@ -41,6 +40,7 @@ pub struct FullDeps<C, P> {
 	pub deny_unsafe: DenyUnsafe,
 }
 
+#[docify::export]
 /// Instantiate all full RPC extensions.
 pub fn create_full<C, P>(
 	deps: FullDeps<C, P>,
@@ -57,6 +57,7 @@ where
 	C::Api: substrate_frame_rpc_system::AccountNonceApi<OpaqueBlock, AccountId, Nonce>,
 	P: TransactionPool + 'static,
 {
+	use substrate_frame_rpc_system::{System, SystemApiServer};
 	let mut module = RpcModule::new(());
 	let FullDeps { client, pool, deny_unsafe } = deps;