Batched transactions (implements NEP#0008) #1140

evgenykuzyakov · 2019-08-06T21:29:51Z

Some tests still fail. It's more to see what files are modified with some auto-replace.

evgenykuzyakov · 2019-08-06T23:19:18Z

Benches (my macbook pro):

running 8 tests
test runtime_send_money                     ... bench:     154,154 ns/iter (+/- 21,694)
test runtime_wasm_bad_code                  ... bench:     175,364 ns/iter (+/- 17,458)
test runtime_wasm_benchmark_10_reads_legacy ... bench:     375,788 ns/iter (+/- 18,445)
test runtime_wasm_benchmark_storage_100     ... bench:     753,850 ns/iter (+/- 116,437)
test runtime_wasm_benchmark_storage_1000    ... bench:   5,224,499 ns/iter (+/- 364,528)
test runtime_wasm_benchmark_sum_1000        ... bench:     274,094 ns/iter (+/- 32,330)
test runtime_wasm_benchmark_sum_1000000     ... bench:   8,004,363 ns/iter (+/- 342,908)
test runtime_wasm_set_value                 ... bench:     284,366 ns/iter (+/- 29,012)

evgenykuzyakov · 2019-08-06T23:27:34Z

Previous benches for comparison:

running 8 tests
test runtime_send_money                     ... bench:     148,466 ns/iter (+/- 7,579)
test runtime_wasm_bad_code                  ... bench:     176,975 ns/iter (+/- 11,976)
test runtime_wasm_benchmark_10_reads_legacy ... bench:     375,113 ns/iter (+/- 66,967)
test runtime_wasm_benchmark_storage_100     ... bench:     742,443 ns/iter (+/- 58,517)
test runtime_wasm_benchmark_storage_1000    ... bench:   5,192,146 ns/iter (+/- 420,130)
test runtime_wasm_benchmark_sum_1000        ... bench:     265,274 ns/iter (+/- 16,762)
test runtime_wasm_benchmark_sum_1000000     ... bench:   8,006,420 ns/iter (+/- 967,673)
test runtime_wasm_set_value                 ... bench:     279,371 ns/iter (+/- 20,418)

chain/chain/src/test_utils.rs

chain/client/src/view_client.rs

core/primitives/src/transaction.rs

core/primitives/src/receipt.rs

core/primitives/src/transaction.rs

ilblackdragon · 2019-08-09T17:26:36Z

test-utils/state-viewer/src/main.rs

@@ -50,10 +49,12 @@ fn kv_to_state_record(key: Vec<u8>, value: DBValue) -> StateRecord {
                StateRecord::Account { account_id: to_printable(&key[1..]), account }
            }
        }
+        /*


ilblackdragon · 2019-08-09T17:26:50Z

test-utils/state-viewer/src/main.rs

@@ -86,9 +87,11 @@ fn print_state_entry(key: Vec<u8>, value: DBValue) {
                to_printable(&from_base64(&value).unwrap())
            );
        }
+        /*


ilblackdragon · 2019-08-09T17:30:37Z

runtime/runtime/src/store.rs

@@ -16,8 +15,7 @@ pub enum StateRecord {
    Contract { account_id: AccountId, code: String },
    /// Access key associated with some account.
    AccessKey { account_id: AccountId, public_key: ReadablePublicKey, access_key: AccessKey },
-    /// Callback.
-    Callback { id: Vec<u8>, callback: Callback },
+    // TODO: DATA


we need it for state dump to work, right?

Yes. Will push it separately

runtime/runtime/src/actions.rs

MaksymZavershynskyi

Some of the gas-related stuff will later go into near-vm-logic.

MaksymZavershynskyi · 2019-08-09T17:40:06Z

core/primitives/src/block.rs

@@ -25,13 +25,10 @@ pub struct BlockHeader {
    /// Height of this block since the genesis block (height 0).
    pub height: BlockIndex,
    /// Hash of the block previous to this in the chain.
-    #[serde(with = "base_format")]


Why we don't use base here anymore?

We've implemented it for basic types. It helps to avoid things like Vec to have a type for base_vec_bytes_format.

MaksymZavershynskyi · 2019-08-09T17:50:04Z

core/primitives/src/crypto/signature.rs

@@ -150,6 +192,25 @@ impl TryFrom<&str> for SecretKey {
    }
 }

+impl Serialize for SecretKey {


This looks a bit weird. Where do we need to serialize the PublicKey without going through the proto? When returning through RPC as JSON? When override Serialize like that we affect any type of serializer, which is kinda fine for now since we probably only serialize it as JSON at the moment, but if someone later in some crate serializes it with bincode (e.g. for cache) it will also go through base encoding which is unnecessary.

Yes, it's currently for serde serialization as json.

MaksymZavershynskyi · 2019-08-09T18:07:16Z

core/primitives/src/types.rs

-/// Identifier for callbacks, used to store storage and refer in receipts.
-pub type CallbackId = Vec<u8>;
+/// Gas is a type for storing amount of gas.
+pub type Gas = u64;


Gas seems fine to me, since we don't have different underlying types for GasUsed, GasBurnt etc (for the contrast for storage we had StorageUsage and StorageUsageChange because one is u64 and another is i64). It is not part of the protobuf or the protocol, so we can always change it later if we find it confusing or our code gets more complex.

runtime/runtime/src/actions.rs

MaksymZavershynskyi · 2019-08-09T18:14:32Z

runtime/runtime/src/actions.rs

+    stake: &StakeAction,
+) {
+    let mut account = account.as_mut().unwrap();
+    let increment = if stake.stake > account.staked { stake.stake - account.staked } else { 0 };


Is it how we stake? If I staked already X and I want to increase my stake to X+Y then I issue a staking transaction with value X+Y instead of Y, correct?

MaksymZavershynskyi · 2019-08-09T18:19:41Z

runtime/runtime/src/config.rs

-    pub add_key: Balance,
-    pub delete_key: Balance,
-    pub delete_account: Balance,
+pub struct TransactionCosts {


This structure might need to go entirely into near-vm-logic.

So this is what I start to wonder. If we decide to charge for every byte of the message (e.g. for args over network). This logic might need to be replicated with the VM, since both of them should produce same Gas cost results.

I suggest we do the following:
For each binding function of VM we define a cost function: F(args, result)->Gas. Which represents the fees that will be applied to call this function, including side-effects like creating receipts. We have this function in near-vm-logic and the runtime crate receives this information as part of used_gas.

Pros:

The logic is encapsulated in near-vm-logic which means it can be used for unit tests in smart contracts. People can write a smart contract, cover it with unit tests and if they did good job with coverage they can expect it to work 100% on the blockchain, because anything not covered by near-vm-* is guaranteed by the rest of engine (like receipts delivery, etc).

Our economics-related stuff is in one place, so we do a good job testing it to prevent all kinds of attacks, secondary markets, etc. We might even succeed formalizing it into an economic model.

Cons:

F(args, result)->Gas will not represent the actual cost of sending a receipt over the network, including serialization etc. However, independently on our architecture we cannot devise a function (whether it is in runtime or near-vm-logic) that can encapsulate all side-effects and costs, so it will always be an approximation. For instance we do not account for temporary storage that cross-contract calls create in the trie because we cannot measure how many nodes deep it will be, etc. So I suggest we always think of this fee as an approximation, and since it is an approximation, why not keep it in near-vm-logic.

MaksymZavershynskyi · 2019-08-11T21:22:59Z

FYI this PR did not fix expensive tests, some of them do not even compile, to see run tests like this:

cargo test --all --features expensive_tests

Expensive tests are broken

MaksymZavershynskyi · 2019-08-11T22:01:54Z

Related: #1156

Co-Authored-By: Maksym Zavershynskyi <35039879+nearmax@users.noreply.github.com>

…core into batched-tx-1

evgenykuzyakov changed the title ~~[WIP] Batched transactions~~ Batched transactions (implements NEP#0008) Aug 6, 2019

evgenykuzyakov marked this pull request as ready for review August 6, 2019 22:51

evgenykuzyakov requested a review from ilblackdragon as a code owner August 6, 2019 22:51

evgenykuzyakov requested review from bowenwang1996 and MaksymZavershynskyi August 6, 2019 22:51

evgenykuzyakov assigned MaksymZavershynskyi Aug 6, 2019

bowenwang1996 reviewed Aug 7, 2019

View reviewed changes

chain/chain/src/test_utils.rs Outdated Show resolved Hide resolved

chain/client/src/view_client.rs Outdated Show resolved Hide resolved

evgenykuzyakov force-pushed the batched-tx-1 branch from c58ab82 to a08667e Compare August 7, 2019 22:24

ilblackdragon reviewed Aug 8, 2019

View reviewed changes

ilblackdragon reviewed Aug 9, 2019

View reviewed changes

runtime/runtime/src/actions.rs Outdated Show resolved Hide resolved

evgenykuzyakov force-pushed the batched-tx-1 branch 2 times, most recently from b1b57cb to ccba8e1 Compare August 9, 2019 17:46

MaksymZavershynskyi previously approved these changes Aug 9, 2019

View reviewed changes

ilblackdragon approved these changes Aug 10, 2019

View reviewed changes

coqbot added the needs: rebase label Aug 11, 2019

Maksym Zavershynskyi added 2 commits August 11, 2019 15:11

Fix base58->base64 encoding

e84db07

Use alice node for RPC tests

1f177d8

MaksymZavershynskyi mentioned this pull request Aug 11, 2019

Wasm -> near-vm-logic & near-vm-runner (Implements NEPS#0009) #1160

Merged

evgenykuzyakov and others added 4 commits August 12, 2019 10:33

Batched transactions (squashed commit)

5846c6c

Update runtime/runtime/src/actions.rs

91888df

Co-Authored-By: Maksym Zavershynskyi <35039879+nearmax@users.noreply.github.com>

Update runtime/runtime/src/actions.rs

67aec5e

Co-Authored-By: Maksym Zavershynskyi <35039879+nearmax@users.noreply.github.com>

Make expensive tests compile

edbd6ab

evgenykuzyakov force-pushed the batched-tx-1 branch from 7247c18 to edbd6ab Compare August 12, 2019 17:49

coqbot removed the needs: rebase label Aug 12, 2019

Merge branch 'fixing_expensive_tests' of github.com:nearprotocol/near…

029fd55

…core into batched-tx-1

evgenykuzyakov merged commit d579d64 into staging Aug 12, 2019

evgenykuzyakov deleted the batched-tx-1 branch August 12, 2019 20:04

weekly-digest bot mentioned this pull request Aug 16, 2019

Weekly Digest (9 August, 2019 - 16 August, 2019) #1178

Closed

frol mentioned this pull request Dec 27, 2019

what is ActionReceipt's input_data_ids ? #1899

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Batched transactions (implements NEP#0008) #1140

Batched transactions (implements NEP#0008) #1140

evgenykuzyakov commented Aug 6, 2019 •

edited

Loading

evgenykuzyakov commented Aug 6, 2019

evgenykuzyakov commented Aug 6, 2019

ilblackdragon Aug 9, 2019

ilblackdragon Aug 9, 2019

ilblackdragon Aug 9, 2019

evgenykuzyakov Aug 10, 2019

MaksymZavershynskyi left a comment

MaksymZavershynskyi Aug 9, 2019

evgenykuzyakov Aug 9, 2019

MaksymZavershynskyi Aug 9, 2019

evgenykuzyakov Aug 9, 2019

MaksymZavershynskyi Aug 9, 2019

MaksymZavershynskyi Aug 9, 2019

evgenykuzyakov Aug 9, 2019

MaksymZavershynskyi Aug 9, 2019

evgenykuzyakov Aug 9, 2019

MaksymZavershynskyi Aug 9, 2019 •

edited

Loading

MaksymZavershynskyi commented Aug 11, 2019

MaksymZavershynskyi commented Aug 11, 2019

Batched transactions (implements NEP#0008) #1140

Batched transactions (implements NEP#0008) #1140

Conversation

evgenykuzyakov commented Aug 6, 2019 • edited Loading

evgenykuzyakov commented Aug 6, 2019

evgenykuzyakov commented Aug 6, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MaksymZavershynskyi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MaksymZavershynskyi Aug 9, 2019 • edited Loading

Choose a reason for hiding this comment

MaksymZavershynskyi commented Aug 11, 2019

MaksymZavershynskyi commented Aug 11, 2019

evgenykuzyakov commented Aug 6, 2019 •

edited

Loading

MaksymZavershynskyi Aug 9, 2019 •

edited

Loading