Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update en exit #257

Merged
merged 2 commits into from
Nov 19, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions docs/guide/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,21 +18,21 @@ We plan to let Venus support the development of storage pool. At the begining, w

After that, through several times of re-design and modification, we finally decide to build it as a distributed storage pool. Distributed storage pool will not completely in charge with any single storage provider, but managed by multiple storage providers, which is more like a storage provider group/union.

The external features of Venus distributed storage pool are similar to other pools. Storage Providers can store their resources in distributed geographic nodes. As long as there is a stable connection to the network, the storage provider can create a node accessible to Venus storage pool; The probability of earning the block reward is proportional to the amount of storage the miner contributes to the Filecoin network, and the final result of separate calculation and overall calculation should be the same: ***n1/p+ n2/p+n3/p......= (n1+n2+n3+.....)/p***, which provides a theoretical basis on the algorithm.
The external features of Venus distributed storage pool are similar to other pools. Storage Providers can store their resources in distributed geographic nodes. As long as there is a stable connection to the network, the storage provider can create a node accessible to Venus storage pool; The probability of earning the block reward is proportional to the amount of storage the storage provider contributes to the Filecoin network, and the final result of separate calculation and overall calculation should be the same: ***n1/p+ n2/p+n3/p......= (n1+n2+n3+.....)/p***, which provides a theoretical basis on the algorithm.

For the storage and retrieval, storage providers can access the network through the venus unified entrance or by themselves. In the design and with the permission from storage providers, venus will select the appropriate node to store data according to specific requirements, so that the data is closer to the real scenario.

1. ***Data storage***: component sharing is supported, and the storage will be gradually migrated to MySQL to ensure the metadata security of Venus through some highly available solutions.

2. ***Shared Modules***: support high availability, ensure the quality of service, reduce the time cost and maintenance cost of miners, which make it easier for miners to join the storage pool. Nowadays, mining is a high-tech activity, but to achieve Filecoin's goal to get the utmost out of storage resources, it is inevitable to lower the threshold to attract more participants.
2. ***Shared Modules***: support high availability, ensure the quality of service, reduce the time cost and maintenance cost of storage providers, which make it easier for storage providers to join the storage pool. Nowadays, mining is a high-tech activity, but to achieve Filecoin's goal to get the utmost out of storage resources, it is inevitable to lower the threshold to attract more participants.

3. ***Combine the right for more block rewards***: miners can integrate resources together and help each other pack messages, so as to avoid the situation of small miners that no one can pack messages.
3. ***Combine the right for more block rewards***: storage providers can integrate resources together and help each other pack messages, so as to avoid the situation of small storage providers that no one can pack messages.

4. ***Improve the TPS of Filecoin network***: if there are multiple blocks at the same height, venus-miner can ensure that the messages in each block are different, as the avoidance of highly-repeated messages when blocks are separated.

5. ***Improve the stability of messages***: messages in storing activities can be separated from nodes and managed by venus-messager, which can track the latest state of the chain to set a relatively reasonable estimated gas parameter, and also supports users to set a message parameter themselves.

6. ***Improve the speed of data access***: the current Filecoin network is more like a pool which is merely for data storage, but as a complete data service, it also needs data retrieval to realize its real value. With sufficient miner users, Venus has more chances to promote the real data application. For example, for data distribution, Venus can distribute the data to some miners' nodes nearest to the customers, which can improve the access speed; some public data can distribute these data to more nodes as much as possible to ensure that access at any location is effective, which can greatly improve the performance of IPFS network and further realize the goal of IPFS, as one of the Web3.0 solutions.
6. ***Improve the speed of data access***: the current Filecoin network is more like a pool which is merely for data storage, but as a complete data service, it also needs data retrieval to realize its real value. With sufficient miner users, Venus has more chances to promote the real data application. For example, for data distribution, Venus can distribute the data to some storage providers' nodes nearest to the customers, which can improve the access speed; some public data can distribute these data to more nodes as much as possible to ensure that access at any location is effective, which can greatly improve the performance of IPFS network and further realize the goal of IPFS, as one of the Web3.0 solutions.

## How Venus works

Expand Down Expand Up @@ -69,16 +69,16 @@ venus is similar to lotus-daemon, which is used to support all nodes. venus is c
1. External signature is supported in the API of creating block (request will be proposed to lotus too).
2. The API of message selection supports the selection of multiple batches of messages at the same time, which is used to select messages when multiple storage providers get the block rewards.
3. Support the centralized authorization access to venus-auth.

Therefore, if you are interested in the mixed deployment of venus / lotus, you need to use Venus project which is compatible to Lotus or pick your own CCS.

### venus-miner

The function of venus-miner is to combine the rights to earn block rewards. The program can configure the location of multiple miners and their sealers and wallets, and generate blocks for these miners at the same time. venus-miner has the following features:
The function of venus-miner is to combine the rights to earn block rewards. The program can configure the location of multiple storage providers and their sealers and wallets, and generate blocks for these storage providers at the same time. venus-miner has the following features:

1. Separate data: because the process of generating blocks needs to access data, and storage provider's sealer may be heterogeneous, but now most miners will use their own codes, so there is a problem about how to access these data to realize joint mining. By venus-miner’s independence on the storage organization mode and completing the proofs by miners, no matter how the miner's sealer organizes the storage, as long as the Computeproof API is implemented, venus-miner can generate the proof through it.
1. Separate data: because the process of generating blocks needs to access data, and storage provider's sealer may be heterogeneous, but now most storage providers will use their own codes, so there is a problem about how to access these data to realize joint mining. By venus-miner’s independence on the storage organization mode and completing the proofs by storage providers, no matter how the miner's sealer organizes the storage, as long as the Computeproof API is implemented, venus-miner can generate the proof through it.
2. Separate private key: when calculating random numbers and signature blocks, all operations about private key are accessed through remote wallet, which can improve the security of mining activities.
3. Improve miners' wincount and TPS: when the miner who uses venus-miner rto get multiple (> 1) rights of block rewards in a period, venus-miner will try to select different messages from the message pool for packaging, which can somehow improve the speed of these messages on the chain, and make the block get more gas premium.
3. Improve storage providers' wincount and TPS: when the miner who uses venus-miner rto get multiple (> 1) rights of block rewards in a period, venus-miner will try to select different messages from the message pool for packaging, which can somehow improve the speed of these messages on the chain, and make the block get more gas premium.

### venus-messenger

Expand All @@ -99,7 +99,7 @@ In terms of functions:

### venus-wallet

venus-wallet is a remote wallet, which can support both Lotus and Venus in terms of protocols. In a typical deployment environment, this module is deployed locally by miners, and the security of assets is guaranteed by certain policy configuration. venus-wallet has the following features:
venus-wallet is a remote wallet, which can support both Lotus and Venus in terms of protocols. In a typical deployment environment, this module is deployed locally by storage providers, and the security of assets is guaranteed by certain policy configuration. venus-wallet has the following features:

1. Private key management function: currently it supports the private key management of asymmetric algorithms BLS and SECP, and it can generate random private key, and uses aes128 to encrypt and store the private key symmetrically whilst supports the private key to sign the data.
2. Signature verification: each signature type has its verification policy, which ensures that the signer won't do deceits the wallet through non-credible signature type.
Expand Down
4 changes: 3 additions & 1 deletion docs/guide/Using-venus-Shared-Modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -698,7 +698,7 @@ Wait till lotus-miner to finish up existing sealing jobs and make sure you have
./lotus-convert -lotus-miner-repo=/root/.lotusminer/ -venus-sealer-repo=/root/.venussealer -taskType=2
```

### More on lotus-convert
:::tip

lotus-convert is a utility tool for migrating from lotus to venus.

Expand All @@ -714,6 +714,8 @@ lotus-convert is a utility tool for migrating from lotus to venus.
./lotus-convert -lotus-miner-repo=/root/.lotusminer/ -venus-sealer-repo=/root/.venussealer -taskType=2
```

:::

## Questions?

Find us on [Slack](https://filecoinproject.slack.com/archives/CEHHJNJS3)!
111 changes: 75 additions & 36 deletions docs/master/Incubation_exit_guide.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,22 @@
## Incubation Exit Guide

After each phase of the incubator is over, miners face the problem of withdrawing. Currently there are lotus and venus implementations of Filecoin mainnet. Therefore, the exit options available to miners are:
There are couple options for storage providers to exit incubation program either voluntarily or after each phase ends.

- Access to venus chain services built by other operators or participate in the second phase of the incubator;
- Access the venus chain service built by yourself;
- Switch back to lotus.
- Join a hosted chain service (venus shared modules) by third party.
- Deploy a chain service (venus shared modules) by yourself.
- Switch back to Lotus.

The above is the optional exit plan for miners at the current stage. Here we explain how to access the venus chain service built by yourself and switch back to the lotus plan one by one.
We will go through option 2 and 3 in this documentation.

### Switch back to your own venus
:::warning

- Build the chain service layer, refer to the document [Chain service construction](./Chain_service_construction.md)
Make sure you go through the documentation carefully before carry out the exit. Find us on Slack if you have questions.

- Modify the `venus-wallet` configuration (`~/.venus_wallet/config.toml`) to point to the local chain service.
:::

### Deploy a chain service

Refer to this [document](https://venus.filecoin.io/guide/How-To-Deploy-MingPool.html) to first deploy your own venus chain service. Modify configuration file (`~/.venus_wallet/config.toml`) of `venus-wallet` to point to the newly deployed chain service.

```toml
[APIRegisterHub]
Expand All @@ -21,7 +25,7 @@ Token = "<AUTH_TOKEN_FOR_ACCOUNT_NAME>"
SupportAccounts = ["<ACCOUNT_NAME>"]
```

- Modify the configuration of `venus-sealer` (`~/.venussealer/config.toml`) to point to the local chain service.
Modify the configuration file of `venus-sealer` (`~/.venussealer/config.toml`) to point to the newly deployed chain service.

```toml
[Node]
Expand All @@ -40,69 +44,104 @@ SupportAccounts = ["<ACCOUNT_NAME>"]
Token = <AUTH_TOKEN_FOR_ACCOUNT_NAME>
```

- Restart `venus-walle` --> `venus-sealer`.

[Reference documents](../guide/Using-venus-Shared-Modules.md)
Restart `venus-walle` and `venus-sealer`.


### Switch back to lotus

- To build a lotus synchronization node, the current chain data is already very large, it is recommended to import the data from the snapshot.
Refer to Lotus [documentation](https://docs.filecoin.io/mine/lotus/#protocol-labs-example-architecture) to build and deploy a Lotus node. Sync the node from a snapshot.

```bash
nohup ./lotus daemon --import-snapshot=https://fil-chain-snapshots-fallback.s3.amazonaws.com/mainnet/minimal_finality_stateroots_latest.car > lotus.log 2>&1 &
$ nohup ./lotus daemon --import-snapshot=https://fil-chain-snapshots-fallback.s3.amazonaws.com/mainnet/minimal_finality_stateroots_latest.car > lotus.log 2>&1 &
```
You can also download the snapshot to the local first, and then `--import-snapshot` points to the local path.
Import addresses under your miner_id (eg. owner, worker, controller addresses and etc.) into lotus;

- Import the wallet address related to miner_id (eg.Owner, Worker, Controller, etc.) into lotus;
```bash
# Export private key from venus-wallet
# export private key from venus-wallet
$ ./venus-wallet export <WALLET_ADDRESS>
Password:

# import to lotus
# import them to lotus
$ ./lotus wallet import
Enter private key:
```

- Wait for the sector currently being sealed to be completed (ProveCommitSector message on the chain);

- Stop venus-sealer and initialize a new repo with lotus-miner. The directory should not be the same as the repo of venus-sealer.
Make sure you have no sealing tasks running, messages properly sent on-chain and enough time before your next windowPost deadline. Then stop venus-sealer and initialize a new repo using lotus-miner.

```bash
TRUST_PARAMS=1 ./lotus-miner init --no-local-storage --actor=<minerID> --sector-size=32G --nosync
```

- Create the `store path` of lotus-miner, which is the same as the store path of venus-sealer, so there is no need to move the permanent storage file.
```bash
# Do not add the `--init` flag, because venus-sealer has been created
./lotus-miner storage attach --store <VENUS_SEALER_STORE_PATH>
```
:::warning

Note that lotus-miner repo must not have the same path as the repo of venus-sealer.

:::

Build lotus-fix utility tool, set nextid and import sectors from venus-sealer.

- Modify the key=/storage/nextid in the metadata to ensure that the new sectorID does not start from 1.
```bash
# Compile lotus-fix in the venus-sealer directory
# Skip if you already compiled ffi
$ git submodule update --init --recursive
$ make deps

# Compile lotus-fix
$ make lotus-fix

# Modify the nextid of lotus-miner
./lotus-fix -lotus-miner-repo=<~/.lotusminer> -sid=<max sector id>
# fix nextid and import sectors from venus-sealer
# change repo path accordingly
$ ./lotus-fix -lotus-miner-repo=/root/.lotusminer/ -venus-sealer-repo=/root/.venussealer -taskType=2
```

- Import the Sector data completed by venus-sealer, this is not necessary, because these can also be found on the chain.
:::tip

lotus-fix is a utility tool for migrating from venus to lotus.

```bash
./lotus-fix -task=import-sector -lotus-miner-repo=/home/litao/.lotusminer -venus-sealer-repo=~/.venussealer
import sectors success.
# taskType=0; manually change nextid
# set nextid to 300 with -sid flag
$ ./lotus-fix -lotus-miner-repo=/root/.lotusminer/ -venus-sealer-repo=/root/.venussealer -taskType=0 -sid=300

# taskType=1; import sectors from venus-sealer to lotus-miner
$ ./lotus-fix -lotus-miner-repo=/root/.lotusminer/ -venus-sealer-repo=/root/.venussealer -taskType=1

# taskType=2; atuo change nextid and import sectors from venus-sealer to lotus-miner
$ ./lotus-fix -lotus-miner-repo=/root/.lotusminer/ -venus-sealer-repo=/root/.venussealer -taskType=2
```
> These two steps need to temporarily stop lotus-miner operation, so it is recommended to operate after init. In addition, if the operation is performed by the root user and lotus-miner is performed by a sub-user (eg. test), file authorization is required.

:::

:::warning

Make sure lotus-miner is not running while using lotus-fix.

:::

:::tip

If executing lotus-fix from root user while lotus-miner is run by a user account, you may need set the following.

```bash
# For example, if your user account name is `test`
chown test:test /home/test/ -R
```

- Make a new sector
:::

Attach `store path` to lotus-miner, which should be the same as the store path of venus-sealer.

```bash
# do not add the `--init` flag
./lotus-miner storage attach --store <VENUS_SEALER_STORE_PATH>
```

Pledge a new sector

```bash
# attach seal path for lotus-miner
$ ./lotus-miner storage attach --init --seal <PATH>

$ nohup ./lotus-miner run > miner.log 2>&1 &
$ ./lotus-miner sectors pledge
```
At this point, your cluster has been switched to lotus operation. For follow-up operations, please refer to lotus related documents. The previously created venus-sealer repo can also be deleted.
At this point, your storage system has been switched to lotus. For follow-up operations, please refer to lotus related documents.
Loading