Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Venus-docs New layout #113

Merged
merged 1 commit into from
Jun 12, 2021
Merged

Venus-docs New layout #113

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/.vuepress/components/RedirectTutorial.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<script>
export default {
mounted() {
this.$router.push("Home.html");
this.$router.push("/Overview.html");
}
};
</script>
Expand Down
60 changes: 30 additions & 30 deletions docs/.vuepress/config.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,11 @@ module.exports = {
}
},
themeConfig: {
logo: '/assets/filecoin-logo.svg',
logo: '/assets/venus-logo-title.svg',
lastUpdated: 'Last Updated',
// Optional options for generating "Edit this page" link
// if your docs are in a different repo from your main project:
docsRepo: 'filecoin-project/docs',
docsRepo: 'filecoin-project/venus-docs',
// if your docs are not at the root of the repo:
docsDir: 'docs',
// if your docs are in a specific branch (defaults to 'master'):
Expand All @@ -63,47 +63,46 @@ module.exports = {
title: 'Venus Filecoin',
description: 'Venus is a Go implementation of the Filecoin Distributed Storage Network.',
nav: [
{text: 'Filecoin Docs', link: 'https://docs.filecoin.io'},
{
text: 'Github', link: 'https://github.com/filecoin-project/venus'
},
],
sidebar: [
{
title: 'Venus Filecoin Tutorial',
collapsable: false,
title: 'Getting-Started',
collapsable: true,
children: [
['Home.md', 'Home'],
['Overview.md', 'Overview'],
['How-To-Deploy-MingPool.md', 'How to deploy mining pool'],
['Using-venus-Shared-Modules.md', 'How to join mining pool'],
['Venus-replace-lotus.md', 'Venus replace Lotus'],
['Tips-Running-In-China.md', 'Tips running in China'],
['venus_load_balancing.md', 'Venus load balancing'],
['How-To-Use-Messager.md', 'How to use messager'],
['Venus-Worker.md', 'How to use Venus Worker'],
['How-To-Use-Wallet-In-Venus.md', 'How to use wallet in venus'],
['Multisig-wallet.md', 'Multisig wallet'],
['Venus wallet.md', 'Venus wallet'],
['Chain.md', 'Chain'],
['Commands.md', 'CLI commands'],
['Getting-Started.md', 'Getting started'],
['How-To-Connect-Network.md', 'How to start network'],
['How-To-Setup_2knet.md', 'How to start a local network'],
['Payment-Channel.md', 'Payment channel'],
['How-To-Deploy-MingPool.md', 'How to deploy a mining pool'],
['Using-venus-Shared-Modules.md', 'How to join a mining pool'],
['Venus-replace-lotus.md', 'Migrate from Lotus'],
['Chain.md', 'Chain management'],
['How-To-Connect-Network.md', 'Connect to different network'],
['Troubleshooting-&-FAQ.md', 'Troubleshooting & FAQ'],
['How-To-Contribute-Docs.md', 'Welcome to Contributing'],
['Tips-Running-In-China.md', 'Tips when running in China'],
['How-To-Contribute-Docs.md', 'Contribute to doc']
]
},
{
title: 'Modules',
collapsable: true,
children: [
['Getting-Started.md', 'Venus daemon'],
['How-To-Use-Messager.md', 'Venus messager'],
['How-To-Use-Wallet-In-Venus.md', 'Venus wallet'],
// ['Venus-wallet.md', 'Venus wallet'],
['Venus-Worker.md', 'Venus worker'],
]
},
{
title: 'Resources',
title: 'Advanced',
collapsable: true,
children: [
['https://docs.filecoin.io', 'Filecoin docs'],
['questions.md', 'Have a question?'],
['Commands.md', 'Venus CLI commands'],
['Tips-Running-In-China.md', 'Tips running in China'],
['Multisig-wallet.md', 'Multisig wallet'],
['venus_load_balancing.md', 'Venus load balancing'],
['Payment-Channel.md', 'Payment channel'],
['How-To-Setup_2knet.md', 'How to start a local dev network'],
]
}
// ]
]
},
'/zh/': {
Expand Down Expand Up @@ -160,3 +159,4 @@ module.exports = {
}
}
}

1 change: 1 addition & 0 deletions docs/.vuepress/public/assets/venus-logo-title.svg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
130 changes: 119 additions & 11 deletions docs/Home.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,127 @@
home: true
---

# Overview
## Preface

### Ahoy, Filecoin community! Welcome to the `Venus` Filecoin tutorial.
Multiple interoperable software implementations are required on the same blockchain. Each implementation has its own set of security issues, but they are not all the same . Therefore, launching cryptocurrency networks with multiple implementations can reduce the catastrophic errors. Venus is one of the implementations of Filecoin network.

Venus is a GoLang implementation of the Filecoin Distributed Storage Network. It was the first Filecoin implementation originially initiated and developed by Protocol Labs, and now is maintained by the IPFS-Force Community. **It is currently under active development.**
Venus (Capital V) refers to one of the general implementations of Filecoin, while venus (v in lowercase) refers to one of the components of Venus. Venus is committed to helping small and medium-sized miners join in the Filecoin ecosystem network with a lower barrier.

Filecoin mainnet is currently live. Visit [network.filecoin.io](https://network.filecoin.io) for all available networks. The tutorials below explore the network using the `Venus` implementation.
Now Venus has realized the support for distributed mine pool of Filecoin through components like venus, venus-sealer, venus-miner, venus-wallet, venus-messager and venus-auth. In the future, Venus will continue its evolvement and jointly promote the open source development of Filecoin ecosystem.

* 🍄 First time using `Venus`? Start with the install and setup steps in [Getting Started](Getting-Started).
* 🤖 Ready to mine? Head over to [Mining Filecoin](How-To-Deploy-MingPool).
* 🌸 PRs, bug reports, and issue suggestions are welcome [Welcome to Contributing](How-To-Contribute-Docs)
* 💔 Need help? Start with [Troubleshooting & FAQ](Troubleshooting-&-FAQ)

### Looking for a introduction to the Filecoin network?
## Why are we doing this

Filecoin currently offers several node implementations including a `Lotus` implementation (also in GoLang) that is currently the recommended default. For a more general introduction to the Filecoin network and underlying concepts that apply across all implementations, please visit the [Filecoin Docs](https://docs.filecoin.io) and [Filecoin Specification](https://spec.filecoin.io).
In June 2020, Protocol Lab launched an RFP to continue the maintenance of go-filecoin by a new community, and IPFSForce applied for and successfully got the project. In October, We renamed go-filecoin as Venus, and continued the maintenance of Venus Project.

Before IPFSForce’s application, maintenance is seriously lacking in Venus project. . In terms of protocols and components, vm,mpool, chain, rpc are all available, but all of them have different problems. It takes us more than three months to gradually fix protocol layer, code layer, and problematic fuctions, making Venus complete the interoperability with Lotus, so that the nodes using Venus can run smoothly on the network. And then we let Venus support calibration, Nerpa and 2K networks.

After completing the above work, we start to set the direction of Venus development.

We plan to let Venus support the development of mining pool. At the start, we decide to realize the expansion from a single miner. It is technically feasible, but the biggest problem is caused by the mechanism of Filecoin proofs. Miners in the Filecoin network have to submit the PoSt periodically, and in order to generate the proof, they need to read the data of a whole partition. If all sectors are distributed in different places, to read the data through the public network will be too time-consuming.

After that, through several times of re-design and modification, we finally realize the current plan, as a distributed mining pool. Distributed mine pool will not completely in charge with any single miner, but managed by multiple miners, which is more like a miner union.

The external features of Venus distributed mining pool are similar to other pools. Miners can store their resources in distributed geographic nodes. As long as there is a place accessible to the network, the miner can create a node accessible to Venus mine pool; The probability of earning the block reward is proportional to the amount of storage the miner contributes to the Filecoin network, and the final result of separate calculation and overall calculation should be the same: ***n1/p+ n2/p+n3/p......= (n1+n2+n3+.....)/p***, which provides a theoretical basis on the algorithm.

On the storage and retrieval of real data, miners can access the network through the venus unified entrance or by themselves. In the design and with the permission from miners, venus will select the appropriate node to store data according to specific requirements, so that the data is closer to the real scenario.

1. ***Data storage***: component sharing is supported, and the storage will be gradually migrated to MySQL to ensure the metadata security of Venus through some highly available solutions.

2. ***Shared components***: support high availability, ensure the quality of service, reduce the time cost and maintenance cost of miners, which make it easier for miners to join the mining pool. Nowadays, mining is a high-tech activity, but to realize Filecoin's goal to get the utmost out of storage resources, it is inevitable to lower the threshold to attract more participants.

3. ***Combine the right for more block rewards***: miners can integrate resources together and help each other pack messages, so as to avoid the situation of small miners that no one can pack messages.

4. ***Improve the TPS of Filecoin network***: if there are multiple blocks at the same height, venus-miner can ensure that the messages in each block are different, as the avoidance of highly-repeated messages when blocks are separated.

5. ***Improve the stability of messages***: messages in mining activities can be separated from nodes and managed by venus-messager, which can track the latest state of the chain to set a relatively reasonable estimated gas parameter, and also supports users to set a message parameter themselves.

6. ***Improve the speed of data access***: the current Filecoin network is more like a pool which is merely for data storage, but as a complete data service, it also needs data retrieval to realize its real value. With sufficient miner users, Venus has more chances to promote the real data application. For example, for data distribution, Venus can distribute the data to some miners' nodes nearest to the customers, which can improve the access speed; some public data can distribute these data to more nodes as much as possible to ensure that access at any location is effective, which can greatly improve the performance of IPFS network and further realize the goal of IPFS, as one of Web3.0 solutions.

## How Venus works

### Architecture

Venus architecture includes venus, venus-miner, venus-sealer, venus-messager, venus-wallet, venus-auth, and venus-market which is still in design. In the classic architecture, venus, venus-miner, venus-auth, venus-messager and venus-market are public components, and venus-sealer and venus-wallet are local components.

Public components can be established by service providers or by multiple miners, and miners only need to run venus-sealer, venus-worker and venus-wallet locally. In fact, all components of Venus are protocol oriented. If miners have sufficient technology, they can also deploy all or part of the public components locally, so the deployment can be more flexible.

The following figure is a typical application scenario in our design.

![./zh/images/venus-arch.png](./zh/images/venus-arch.png)

The following contents show the relationship between components, which are similar in function, but with more powerful expanded functions in details and uses.

* venus-auth --> token authentication service extend lotus jwt
* venus-miner --> lotus-miner block produce part
* venus-sealer --> lotus-miner sealer scheduler part/wd
* venus-worker --> lotus-worker
* venus daemon --> lotus daemon
* venus-wallet -> lotus wallet part
* venus-messager --> lotus mpool

### venus-auth

venus-auth is a component used for authorization. In a typical deployment environment, venus-auth is used to provide authorization services to venus, venus-miner and venus-messager. Local nodes need to be authorized to have access to the shared components. Based on the authorization, venus-auth will expand its functions, such as security control, including access restriction and blacklist.

### venus

venus is similar to lotus-daemon, which is used to realize all nodes. venus is connected to the Filecoin blockchain network to provide data support for Venus. venus can be used as an ordinary node. It has all the necessary functions of blockchain nodes, such as chain, mpool and wallet. The APIs of venus are almost compatible with those of lotus (you can mention the issue of incompatible APIs). On venus, you can create private keys, query assets, transfer funds, etc. In a typical deployment environment, venus only provides data query service on the basic chain without private key restoration. The block data needed by slashfilter is stored in the database, and the node itself is stateless. In this case, venus node can realize its high availability through nginx reverse proxy.

There are some differences between nodes of venus and lotus:

1. External signature is supported in the API of creating block (pr will be proposed to lotus later).
2. The API of message selection supports the selection of multiple batches of messages at the same time, which is used to select messages when multiple miners get the block rewards.
3. Support the centralized authorization access to venus-auth.

Therefore, if you are interested in the mixed deployment of venus / lotus, you need to use our Venus project which is compatible to Venus or pick your own CCS.

### venus-miner

The function of venus-miner is to combine the rights of block rewards. The program can configure the location of multiple miners and their sealers and wallets, and generate blocks for these miners at the same time. venus-miner has the following features:

1. 1.Separate data: because the process of generating blocks needs to access data, and miner's sealer may be heterogeneous, but now most miners will use their own codes, so there is a problem about how to access these data to realize joint mining. By venus-miner’s independence on the storage organization mode and completing the proofs by miners, no matter how the miner's sealer organizes the storage, as long as the Computeproof API is implemented, venus-miner can generate the proof through it.
2. Separate private key: when calculating random numbers and signature blocks, all operations about private key are accessed through remote wallet, which can improve the security of mining activities.
3. Improve miners' income and TPS: when the miner who uses venus-miner rto get multiple (> 1) rights of block rewards in a period, venus-miner will try to select different messages from the message pool for packaging, which can somehow improve the speed of these messages on the chain, and make the block get more probable premium.

### venus-messager

The goal of venus-messager is to help messages on chain more efficiently, and to flexibly control the timing of messages on chain to reduce gas consumption and control the data traffic, etc. It includes remote wallet management, address management and message management.

1. Wallet management: venus-messager connects multiple wallets and scans these addresses into the list.
2. Address management: the major purpose is to manage the nonce value to ensure that the value can be assigned in the correct order.
3. Message management: it is divided into three parts, namely "message receiving and saving", "message selection and pushing" and "tracking the status of message on the chain".

In terms of functions:

1. Support remote wallet: one messager supports multiple wallets to manage their message separately.
2. Support local storage of SQLite and remote storage of MySQL for more secure and stable storage.
3. Dynamic Fill: gas related parameters and nonce are to be filled out when sending a message on chain according to gas and push policy, to make sure the gas-estimation and other setting are valid.
4. Maintain message status, including whether the message is chained and replaced and save the results of the execution.
5. Multi-point message delivery (directly to the blockchain network with libp2p, push to the node by Mpool API), to make sure that messages are propagated over the network.
6. Flexible config includes gas estimate, msg push strategy, etc.

![./zh/images/messager.png](./zh/images/messager.png)

### venus-wallet

venus-wallet is a remote wallet, which can support both Lotus and Venus in terms of protocols. In a typical deployment environment, this component is deployed locally by miners, and the security of assets is guaranteed by certain policy configuration. venus-wallet has the following features:

1. Private key management module: currently it supports the private key management of asymmetric algorithms BLS and SECP, and it can generate random private key, and uses aes128 to encrypt and store the private key symmetrically whilst supports the private key to sign the data.
2. Signature verification: each signature type has its verification policy, which ensures that the signer won't do deceits the wallet through non-credible signature type.
3. Signature tactics module: for the Lotus and Venus implementation of Filecoin, the module is created to manage 10 existing data structures and more than 60 signature types in message data structure, and configure various combination methods on demand to bind private key signature rules, and then integrate several different private key signature rules and authorize them to external components for further usage.

### venus-sealer + venus-worker

venus-sealer continues the basic features of the mining part of lotus. We separate the mining related code from the sealer to form venus-sealer. The differences are:

1. Access to the messager system, and all messages are managed in the messager.
2. Storage is separated from the badger, and SQLite is used to save metadata, sector state, machine information, scheduling and related results. In the future, MySQL storage will be further supported to ensure data security.

## Some ideas for the future

The following plans mainly focus on:
1. Follow up the latest Filecoin network.
2. Start the venus-market project and improve the entire venus system.
3. Optimize venus-sealer scheduling mode and storage (in planning).
4. Improve the availability of components, continuously test components, fix some potential problems, ensure the stability and security of component services, and simplify the deployment mode.
5. Optimize the log system to facilitate problem query and tracking.