Contracts module rejig

[pepyakin]

I started this branch as working on PUTCODE model, described here. After some initial work was done I realized that I can reliably test that code, because all of the testing was done in top-down approach, similar how other modules are doing it. Tests were super fragile, they tested too much, even though the system was split in two parts (one was able to test wasm bindings separately). If that weren't enough all tests had asserts on how much a particular operation spent gas which was merely a single number which changed often.

Because of that I decided to go all in and do a few refactorings to the codebase.

Adding a Vm seam

First, I added a seam between vm and executive layers, so a few of new traits was introduced: Loader and Vm. In essense, they allow you to switch an implementation of a VM with something other that can call interact with the outside world via Ext.
It would be kind of cool to add a feature to run other kind of bytecode, say, EVM, but I didn't dig into. The main purpose of this change is to allow to use a closure executor as a VM (which uses closures instead of bytecode). This allows testing code for more closer and easier interaction with the environment.

For example, consider this test in the current version which is written in raw wasm:

substrate/srml/contract/src/tests.rs

Lines 800 to 853 in 754f28d

	const CODE_INPUT_DATA: &'static str = r#"
	(module
	(import "env" "ext_input_size" (func $ext_input_size (result i32)))
	(import "env" "ext_input_copy" (func $ext_input_copy (param i32 i32 i32)))
	(import "env" "memory" (memory 1 1))
	(func (export "call")
	(block $fail
	;; fail if ext_input_size != 4
	(br_if $fail
	(i32.ne
	(i32.const 4)
	(call $ext_input_size)
	)
	)
	(call $ext_input_copy
	(i32.const 0)
	(i32.const 0)
	(i32.const 4)
	)
	(br_if $fail
	(i32.ne
	(i32.load8_u (i32.const 0))
	(i32.const 0)
	)
	)
	(br_if $fail
	(i32.ne
	(i32.load8_u (i32.const 1))
	(i32.const 1)
	)
	)
	(br_if $fail
	(i32.ne
	(i32.load8_u (i32.const 2))
	(i32.const 2)
	)
	)
	(br_if $fail
	(i32.ne
	(i32.load8_u (i32.const 3))
	(i32.const 3)
	)
	)
	(return)
	)
	unreachable
	)
	)
	"#;

with this code in the proposed PR:

let input_data_ch = loader.insert(|ctx| {
    assert_eq!(ctx.input_data, &[1, 2, 3, 4]);
    VmExecResult::Ok
});

There is a one well-known gotcha with this mocking approach: when you mock too much you don't test anything. To avoid that I had to specify contract between vm and executive layer in more detail, and in particular encoded some invariants in type system (for example, instead of passing a mutable pointer to a &mut output_data for direct write of return data of the callee contract to the scratch buffer of the caller, we now use EmptyOutputBuf/OutputBuf/VmExecResult types which you pass by value now).

This is only the first step. I can see opportunities for upgrading this approach by adding some macro magic for initial setup (which is quite boilerplately right now).

Smart Gas Metering

It was really hard to test gas metering. The strategy was like this

substrate/srml/contract/src/tests.rs

Lines 604 to 610 in 754f28d

	// 11 - value sent with the transaction
	// (3 * 129) - gas spent by the init_code.
	// (3 * 175) - base gas fee for create (175) (top level) multipled by gas price (3)
	// ((21 / 3) * 3) - price for contract creation
	assert_eq!(
	Balances::free_balance(&0),
	100_000_000 - 11 - (3 * 129) - (3 * 175) - ((21 / 3) * 3)

after the execution of a contract you check how much gas was spent (or rather, how much balance units was refunded). You express the forumula as an arithmetic expression that describes how much each component consumed after adjust for gas.

This was really hard to work with because every time something changes you have to figure out which component exactly has changed. Changes from wasm code was especially hard. Changes from wasm code constructors (basically wasm code wrapped in a wasm code) was even harder.

The solution is to require every single charging for gas to provide token, purpose of which is two-fold:

1. It provides a nominal type which you can use then for matching at the testing time.
2. Exact gas calculation. So instead of calculating how much gas to charge you just use the right token and it will calculate the gas for you, thus achieving separation of concerns.

Instead of code like this littered every where

substrate/srml/contract/src/vm/runtime.rs

Lines 342 to 345 in 754f28d

	let data_len_in_gas = <<E::T as Trait>::Gas as As<u64>>::sa(data_len as u64);
	let price = (ctx.schedule.return_data_per_byte_cost)
	.checked_mul(&data_len_in_gas)
	.ok_or(sandbox::HostError)?;

you declare token once

#[cfg_attr(test, derive(Debug, PartialEq, Eq))]
#[derive(Copy, Clone)]
pub enum RuntimeToken {
...
    /// The given number of bytes is read from the sandbox memory.
    ReadMemory(u32),
...
}

impl<T: Trait> Token<T> for RuntimeToken {
    type Metadata = Schedule<T::Gas>;

    fn calculate_amount(&self, metadata: &Schedule<T::Gas>) -> Option<T::Gas> {
        let value = match *self {
...
            ReadMemory(byte_count) => metadata
                .sandbox_data_read_cost
                .checked_mul(&<T::Gas as As<u32>>::sa(byte_count))?,
...
        };

        Some(value)
    }
}

and then use it like that

gas_meter.charge(&schedule, RuntimeToken::ReadMemory(bytes))?;

Other changes

• the environment definition macros were adapted for
• put code model also opened up a way to reuse instrumented code. Fixes srml-contract: Reuse instrumented code #1040

[pepyakin]

@bkchr @gnunicorn I think this is ready for another look!

[gnunicorn]

Minor code doc fixes needed (suggestions supplied)

[pepyakin]

@gnunicorn thanks for the suggestions, I really appreciate it!

[gavofyork]

@bkchr any other remarks before i merge?

[bkchr]

I'm okay @pepyakin just update the wasm files and this can be merged.