Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: add testing framework document #142

Merged
merged 1 commit into from
Feb 24, 2023
Merged

docs: add testing framework document #142

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions tests/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# I. Why we need to setup an automated testing framework?
In many cases, we have observed that unit test isn't enough to guarantee that the whole system will not break. We simply got lucky when Ed found a problem through his manual system testing that may not always happen.

In some cases, not even the unit test is correct due to writer misunderstanding of required logic.

# II. Framework
1. Standarize writing unit test (we can even go further with a ci to check). A PR should include:
* feature specification (in written markdown file in a folder called feature-request) (Gherkin language)
* unit test write

2. a full - fledged e2e test flow through ci that does following flow
* upgrade testing:
* for chain (if exists, else use current terrad): do an upgrade gov test to newer chain binary
* for wasm (if there are changes to wasm, else use current wasm files): change to newer wasm files
* regression testing (change to system should not break existing one)
* backend:
* chain should run normal
* ibc should work normal
* wasm should run normal
* state sync should run normal
* frontend: api testing of most important transaction types and query
* fuzz testing:
* some nice background reading: https://github.com/osmosis-labs/osmosis/blob/main/simulation/ADR.md
* we need to care only about events that invoke a state change, we then create randomized request to these events:
* transaction: create randomized request and broadcast multiple times
* begin block: randomized environment condition that invokes a begin block logic (Ex: system time, number of validators, ...)
* end block: randomized environment condition that invokes an end block logic (Ex: system time, number of validators, ...)
* init genesis: create randomized genesis state

It is hard to invoke a begin, end block logic without first understanding what environment condition it relies on.