Skip to content

Commit

Permalink
doc: nearcore migration guide to 1.0.0
Browse files Browse the repository at this point in the history
  • Loading branch information
dj8yf0μl committed Aug 16, 2023
1 parent 80d5ba1 commit 0d346f1
Show file tree
Hide file tree
Showing 2 changed files with 354 additions and 0 deletions.
2 changes: 2 additions & 0 deletions borsh/src/lib.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -81,10 +81,12 @@ pub mod ser;

pub use de::BorshDeserialize;
pub use de::{from_reader, from_slice};

#[cfg(derive_schema)]
pub use schema::BorshSchema;
#[cfg(derive_schema)]
pub use schema_helpers::{try_from_slice_with_schema, try_to_vec_with_schema};

pub use ser::helpers::{to_vec, to_writer};
pub use ser::BorshSerialize;

Expand Down
352 changes: 352 additions & 0 deletions migration_guides/v0.10.2_to_v1.0.0_nearcore.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,352 @@
# `0.10.2` -> `1.0.0-alpha.3` nearcore upgrade *migration guide*

The link to `nearcore` pr is [chore: update borsh dependency](https://github.com/near/nearcore/pull/9432)

Steps:

## 1. update dependencies in `nearcore` workspace `Cargo.toml`

```diff
-borsh = { version = "0.10.2", features = ["rc"] }
+borsh = { version = "1.0.0-alpha.3", features = ["derive", "rc"] }
```

## 2. next we encounter error in `near-primitives` package:

```bash
1 error[E0433]: failed to resolve: could not find `maybestd` in `borsh` ▐
--> core/primitives/src/receipt.rs:1:19 ▐
| ▐
1 | use crate::borsh::maybestd::collections::HashMap; ▐
| ^^^^^^^^ could not find `maybestd` in `borsh` ▐
```

`maybestd` has moved to a `__private` package in `borsh`, and is not supposed to be
accessed directly now besides from within code, derived in `borsh` traits implementations.

As `near-primitives` crate is not supposed to be used in `no_std` context, we can
replace import with standard collections `HashMap`:

```diff
diff --git a/core/primitives/src/receipt.rs b/core/primitives/src/receipt.rs
index 30af36fb9..d5a6632ed 100644
--- a/core/primitives/src/receipt.rs
+++ b/core/primitives/src/receipt.rs
@@ -1,11 +1,11 @@
-use crate::borsh::maybestd::collections::HashMap;
+use std::collections::HashMap;
```

Otherwise, we would've imported from `hashbrown`:

```diff
-use crate::borsh::maybestd::collections::HashMap;
+#[cfg(feature = "std")]
+use std::collections::HashMap;
+#[cfg(not(feature = "std"))]
+use hashbrown::HashMap;
```

## 3. next we encounter a bunch of similar errors in `near-primitives` with `#[borsh_init(...)]`:

```bash
1 error: cannot find attribute `borsh_init` in this scope ▐
--> core/primitives/src/block_header.rs:267:3 ▐
| ▐
267 | #[borsh_init(init)] ▐
| ^^^^^^^^^^ help: a derive helper attribute with a similar name exists: `borsh_skip`▐
```

The syntax of this attribute has changed. We change all of these occurencies according to
`#[borsh(init=<initialization_method>)]` syntax. The following diff is shortened to first and last
occurencies:

```diff
diff --git a/core/primitives/src/block_header.rs b/core/primitives/src/block_header.rs
index 38491b52c..84ab48238 100644
--- a/core/primitives/src/block_header.rs
+++ b/core/primitives/src/block_header.rs
@@ -266,3 +266,3 @@ impl ApprovalMessage {
#[derive(BorshSerialize, BorshDeserialize, serde::Serialize, Debug, Clone, Eq, PartialEq)]
-#[borsh_init(init)]
+#[borsh(init=init)]
pub struct BlockHeaderV1 {
...
diff --git a/core/primitives/src/transaction.rs b/core/primitives/src/transaction.rs
index 912120b56..2de7a1d52 100644
--- a/core/primitives/src/transaction.rs
+++ b/core/primitives/src/transaction.rs
@@ -58,3 +58,3 @@ impl Transaction {
)]
-#[borsh_init(init)]
+#[borsh(init=init)]
pub struct SignedTransaction {
```

## 4. next we encounter a large number of similar syntax errors in `near-primitives` package

```bash
1 error: cannot find attribute `borsh_skip` in this scope ▐
--> core/primitives/src/transaction.rs:196:7 ▐
| ▐
196 | #[borsh_skip] ▐
| ^^^^^^^^^^ ▐
▐
```

We change all of these occurencies according to
`#[borsh(skip)]` syntax. The following diff is shortened to first and last
occurencies:

```diff
diff --git a/core/primitives/src/block_header.rs b/core/primitives/src/block_header.rs
index 84ab48238..6514f8222 100644
--- a/core/primitives/src/block_header.rs
+++ b/core/primitives/src/block_header.rs
@@ -279,3 +279,3 @@ pub struct BlockHeaderV1 {
/// Cached value of hash for this block.
- #[borsh_skip]
+ #[borsh(skip)]
pub hash: CryptoHash,
...
diff --git a/core/primitives/src/transaction.rs b/core/primitives/src/transaction.rs
index 2de7a1d52..f3ac54ba8 100644
--- a/core/primitives/src/transaction.rs
+++ b/core/primitives/src/transaction.rs
@@ -62,5 +62,5 @@ pub struct SignedTransaction {
pub signature: Signature,
- #[borsh_skip]
+ #[borsh(skip)]
hash: CryptoHash,
- #[borsh_skip]
+ #[borsh(skip)]
size: u64,
...
```



## 5. next we encounter 2 errors in `near-primitives` package similar to those in point 2.:

```bash
1 error[E0433]: failed to resolve: could not find `maybestd` in `borsh`
--> core/primitives/src/action/delegate.rs:119:41
|
119 | fn deserialize_reader<R: borsh::maybestd::io::Read>(
| ^^^^^^^^ could not find `maybestd` in `borsh`
2 error[E0433]: failed to resolve: could not find `maybestd` in `borsh`
--> core/primitives/src/action/delegate.rs:121:50
|
121 | ) -> ::core::result::Result<Self, borsh::maybestd::io::Error> {
| ^^^^^^^^ could not find `maybestd` i
n `borsh`
```
As `near-primitives` crate is not supposed to be used in `no_std` context, we can
replace import with `std::io`:
```diff
diff --git a/core/primitives/src/action/delegate.rs b/core/primitives/src/action/delegate.rs
index 25db73022..ebd009a44 100644
--- a/core/primitives/src/action/delegate.rs
+++ b/core/primitives/src/action/delegate.rs
@@ -14,3 +14,3 @@ use near_primitives_core::types::{AccountId, Nonce};
use serde::{Deserialize, Serialize};
-use std::io::{Error, ErrorKind};
+use std::io::{Error, ErrorKind, Read};

@@ -118,5 +118,5 @@ mod private_non_delegate_action {
impl borsh::de::BorshDeserialize for NonDelegateAction {
- fn deserialize_reader<R: borsh::maybestd::io::Read>(
+ fn deserialize_reader<R: Read>(
rd: &mut R,
- ) -> ::core::result::Result<Self, borsh::maybestd::io::Error> {
+ ) -> ::core::result::Result<Self, Error> {
match u8::deserialize_reader(rd)? {
```
Otherwise, we would've imported from `borsh::nostd_io`:
```diff
+#[cfg(feature = "std")]
+use std::io;
+#[cfg(not(feature = "std"))]
+use borsh::nostd_io as io;
```
## 6. next we encounter an error with `BorshDeserialize` trait derivation:
```bash
1 error[E0277]: the trait bound `&T: borsh::BorshDeserialize` is not satisfied
--> core/primitives/src/signable_message.rs:58:26
|
58 | #[derive(BorshSerialize, BorshDeserialize)]
| ^^^^^^^^^^^^^^^^ the trait `borsh::BorshDeserialize` is not i
mplemented for `&T`
```
on
```rust
/// A wrapper around a message that should be signed using this scheme.
///
/// Only used for constructing a signature, not used to transmit messages. The
/// discriminant prefix is implicit and should be known by the receiver based on
/// the context in which the message is received.
#[derive(BorshSerialize, BorshDeserialize)]
pub struct SignableMessage<'a, T> {
pub discriminant: MessageDiscriminant,
pub msg: &'a T,
}
```
On version change `0.10.3` -> `1.0.0-alpha.3` bounds derivation in `borsh` has changed:
From bounds on the types of the fields:
```rust
impl<'a, T> borsh::de::BorshDeserialize for SignableMessage<'a, T>
where
MessageDiscriminant: borsh::de::BorshDeserialize,
&'a T: borsh::de::BorshDeserialize,
```
to bounds on type parameters, encountered in fields:
```rust
impl<'a, T> borsh::de::BorshDeserialize for SignableMessage<'a, T>
where
T: borsh::de::BorshDeserialize,
```
We could potentially patch the bounds on struct to make it compile:
```rust
#[derive(BorshSerialize, BorshDeserialize)]
pub struct SignableMessage<'a, T> {
pub discriminant: MessageDiscriminant,
#[borsh(bound(deserialize="&'a T: borsh::de::BorshDeserialize"))]
pub msg: &'a T,
}
```
which would transform into following bould on trait's implementation:
```rust
where
&'a T: borsh::de::BorshDeserialize,
```
But the real issue here is that `borsh` doesn't have a generic implementation
of `BorshDeserialize` for `&'a T`, where `T: borsh::de::BorshDeserialize` (nor did it have it in 0.10.2 version),
and that the derived `BorshDeserialize` wasn't used (and it couldn't be for such a field's type).
So the right change is to remove `BorshDeserialize` derive from the struct:
```diff
diff --git a/core/primitives/src/signable_message.rs b/core/primitives/src/signable_message.rs
index efdd489ac..db97eb1fd 100644
--- a/core/primitives/src/signable_message.rs
+++ b/core/primitives/src/signable_message.rs
@@ -57,3 +57,3 @@ pub struct MessageDiscriminant {
/// the context in which the message is received.
-#[derive(BorshSerialize, BorshDeserialize)]
+#[derive(BorshSerialize)]
pub struct SignableMessage<'a, T> {
```
## 7. next we encounter an error in `near-network` package:
```rust
1 error: You have to specify `#[borsh(use_discriminant=true)]` or `#[borsh(use_discriminant=false)]` for all enums with explicit discriminant ▐
--> chain/network/src/types.rs:56:10 ▐
| ▐
56 | pub enum ReasonForBan { ▐
| ^^^^^^^^^^^^ ▐
```
```rust
#[derive(borsh::BorshSerialize, borsh::BorshDeserialize, Debug, Clone, PartialEq, Eq, Copy)]
pub enum ReasonForBan {
None = 0,
BadBlock = 1,
BadBlockHeader = 2,
HeightFraud = 3,
BadHandshake = 4,
BadBlockApproval = 5,
Abusive = 6,
InvalidSignature = 7,
InvalidPeerId = 8,
InvalidHash = 9,
InvalidEdge = 10,
InvalidDistanceVector = 11,
Blacklisted = 14,
}
```
We fix it with `#[borsh(use_discriminant=false)]` to preserve the behaviour of borsh before
1.0 release which serialized `ReasonForBan::Blacklisted` as 12 instead of 14
(borsh 0.10 and older [ignored explicit discriminant values in enum definitions](https://github.com/near/borsh-rs/issues/137)):
```diff
diff --git a/chain/network/src/types.rs b/chain/network/src/types.rs
index b2dd97c32..ea2d67f2d 100644
--- a/chain/network/src/types.rs
+++ b/chain/network/src/types.rs
@@ -55,2 +55,3 @@ pub struct KnownProducer {
#[derive(borsh::BorshSerialize, borsh::BorshDeserialize, Debug, Clone, PartialEq, Eq, Copy)]
+#[borsh(use_discriminant=false)]
pub enum ReasonForBan {
```
## 8. change in behaviour, unit test error in ci in `near-primitives` package
Assertion fails:
```rust
#[test]
fn test_delegate_action_deserialization() {
// Expected an error. Buffer is empty
assert_eq!(
NonDelegateAction::try_from_slice(Vec::new().as_ref()).map_err(|e| e.kind()),
Err(ErrorKind::InvalidInput)
);
```
```bash
--- STDERR: near-primitives action::delegate::tests::test_delegate_action_deserialization ---
thread 'action::delegate::tests::test_delegate_action_deserialization' panicked at 'assertion failed: `(left == right)`
left: `Err(InvalidData)`,
right: `Err(InvalidInput)`', core/primitives/src/action/delegate.rs:172:9
```
The `ErrorKind` in error in `borsh` has changed, so we apply the following diff:
```diff
diff --git a/core/primitives/src/action/delegate.rs b/core/primitives/src/action/delegate.rs
index ebd009a44..80a0475b6 100644
--- a/core/primitives/src/action/delegate.rs
+++ b/core/primitives/src/action/delegate.rs
@@ -173,3 +173,3 @@ mod tests {
NonDelegateAction::try_from_slice(Vec::new().as_ref()).map_err(|e| e.kind()),
- Err(ErrorKind::InvalidInput)
+ Err(ErrorKind::InvalidData)
);
```
And there's also a similar error in
```bash
--- STDERR: near-store tests::test_save_to_file ---
thread 'tests::test_save_to_file' panicked at 'assertion failed: `(left == right)`
left: `InvalidInput`,
right: `InvalidData`', core/store/src/lib.rs:1096:9
```
with similar fix.
## 9. errors similar to previous ones, in `near-store`, `near-network` and `near-state-viewer` packages
There was a bunch of `borsh::maybestd` imports, which got replaced by their direct from-`std` counterparts.

0 comments on commit 0d346f1

Please sign in to comment.