go-spacemesh<>svm API

[dshulyak]

Full API

type TxHeader struct {
	Principal   Address
	Template    Address
	Method      uint8
	Nonce       Nonce
	LayerLimits LayerLimits
	MaxGas      uint64
	GasPrice    uint64
	MaxSpend    uint64
}

type ValidationRequest interface {
	Parse() (*types.TxHeader, error)
	Verify() bool
}

type VM interface {
	Validation(types.RawTx) system.ValidationRequest
	Apply(types.LayerID, []types.RawTx, []types.AnyReward) ([]types.TransactionID, error)
    
	GetStateRoot() (types.Hash32, error)
	GetLayerStateRoot(types.LayerID) (types.Hash32, error)
	GetLayerApplied(types.TransactionID) (types.LayerID, error)
	GetAllAccounts() ([]*types.Account, error)
	GetBalance(types.Address) (uint64, error)
	GetNonce(types.Address) (types.Nonce, error)
	Revert(types.LayerID) (types.Hash32, error)
}

This is not a final API, in case if it will be implemented in svm this API will have to output rewards and execution results after Apply (maybe more).

cons:

1. can't use single storage for all state. consistency problems are guaranteed.
   they actually follow straight from the API
2. scope creep

• rewards computation is moved to svm. any changes in rewards will have to modify rust code.
• commitment computation and storage is moved to svm. any interaction with commitment data structures will go through go<>rust boundary. so for example light client sync will have to go through this boundary as well

ultimately it will be painful to work on the codebase where half of the application is implemented in rust and the other half in go. the only way to make it work is to reduce the scope of svm api to the bare minimum.

pros:

1. no limitations on the rust implementation

External storage API

type ValidationRequest interface {
	Parse() (*types.TxHeader, error)
	Verify() bool
} 

type Result struct {
    Updated  []Address
    Results  []ExecutionResult
}

type ExecutionResult struct {
    ID types.TransactionID // 32 bytes
    Status uint8 // skipped?/failed/applied
    GasUsed uint64
    // and probably couple of other fields.
}

type Storage interface {
    GetAccount(address types.Address) types.Account
    // and other methods that are necessary for svm
}

type VM interface {
	Validation(types.RawTx) system.ValidationRequest
	Apply(Storage, types.LayerID, []types.RawTx) (Result, error)
}

Storage API is implemented in the go-spacemesh and exposed to svm using C api. This is also the current approach taken by filecoin fvm (see https://github.com/filecoin-project/filecoin-ffi/tree/master/rust/src/fvm), but they also have other limitations, namely storage backend is implemented in golang.

cons:

1. harder to test svm, as it needs storage implementation for internal tests
2. state reads will cross go boundary which may be expensive (?)

pros:

1. no problems with state consistency
2. no scope creep
3. simple to implement

Same as previous but with the pointer to the storage

type Storage struct {
    Ptr *C.sqlite3
}

On the rust side connection needs to be created by using ffi from this Ptr. See https://github.com/rusqlite/rusqlite/blob/master/src/inner_connection.rs#L19.

Note that it should work with any backend in C (or in rust) not just sqlite.

pros:

1. svm won't have problems with a backend for tests
2. svm doesn't cross go boundary to read the state
3. no state consistency issues
4. no scope creep

cons:

1. same storage lib will have to be linked with svm and go-spacemesh
2. rust bindings will have to be adjusted so that connection can be initialized from existing pointer to sqlite3 struct

the bottomline is that this approach requires more engineering effort than the previous two.

[lrettig]

Quick thought: it's likely that we want the nonce update to happen inside the VM (rather than inside the node). The reason is a bit arcane but has to do with the fact that different templates can choose to use the nonce/bitfield bytes differently and it's probably more efficient to do the update inside the VM. Will share more on a comprehensive nonce plan soon. I don't know if this has any impact on the above API: nonce check can happen inside Validation(), update can happen inside Apply(), and the node still needs to be able to call GetNonce() to manage the mempool and sort txs into blocks.

We need to finalize the signature for GetNonce(), which likely needs to return both a nonce and bitfield value - more on this soon too.