-
Notifications
You must be signed in to change notification settings - Fork 5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add documentation for testing and CI #3532
Merged
Merged
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
ca80264
Add documentation for testing and CI
cgewecke 8f1a4ec
Add implementation details section to TESTING.md
cgewecke 834cb78
Clarify surge site test check requirements
cgewecke 7b0ceed
Misc rewordings and clarifications
cgewecke 58819c2
Merge branch '1.x' into issue/3528
cgewecke 0460c18
Merge branch '1.x' into issue/3528
cgewecke File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
# Guide to Web3's tests and CI | ||
|
||
Web3 is used in Node.js and browser contexts to interact with a wide variety of clients. Its tests | ||
try to cover as much of this domain as possible. | ||
|
||
If you're looking for a fixture, test pattern or common execution context to validate a change, you should be able find it in the existing test suite. (Caveats include Parity / Quorum clients and MetaMask specific tests.) | ||
|
||
## Required Tests | ||
|
||
These should pass for PRs to merge: | ||
|
||
| Test type | npm command | Example | Description | CI Only | | ||
| --------- | --------------- | ------ | ----------- | ----- | | ||
| unit | test | [eth.accounts.sign.js][1] | For discrete pieces of logic | | ||
| integration | test:e2e:clients | [e2e.contract.events.js][2] | Tests using geth and ganache-cli, (insta-mining and interval mining.) Easy to write and good for modeling complex use-cases | | ||
| browser | test:e2e:browsers | | The integration tests run in a headless browser using web3.min.js (browserified, vs. ganache-cli) | | ||
| typescript | dtslint | -- | TS type definitions tests | | ||
| dependencies | depcheck | -- | Verifies every dependency is listed correctly in the module package | | ||
| bundle | test:e2e:min | [e2e.minified.js][3] | Verifies minified bundle loads in a headless browser *without* being webpacked / browserified | :white_check_mark: | | ||
| cdn | test:e2e:cdn | [e2e.cdn.sh][4]| Visual inspection check: publishes an (un-webpacked) site that uses web3.min.js at http://sudden-playground.surge.sh/ | :white_check_mark: | | ||
| windows | -- | [e2e.windows.sh][5] | Verifies Web3 installs on Windows OS / Node 12 and can connect to Infura over wss and https | :white_check_mark: | | ||
|
||
|
||
## Optional Tests | ||
|
||
CI also has tests that install Web3's state at an arbitrary commit in an external real-world project and run *their* unit tests with it. This strategy is borrowed from ethereum/solidity which checks latest Solidity against OpenZeppelin and others to keep abreast of how local changes might affect critical projects downstream from them. | ||
|
||
Examples include: | ||
+ [e2e.mosaic.sh][8]: ~300 unit tests for [a Solidity project built with Buidler & @truffle/contract][9] | ||
+ [e2e.ganache.core.sh][9]: ~600 unit tests for [a widely used JS testrpc][11] | ||
|
||
These tests are "allowed failures". They're: | ||
+ a pre-publication sanity check that discovers how Web3 performs in the wild | ||
+ useful for catching problems which are difficult to anticipate | ||
+ exposed to failure for reasons outside of Web3's control, ex: when fixes here surface bugs in the target. | ||
|
||
|
||
[8]: https://github.com/ethereum/web3.js/blob/1.x/scripts/e2e.mosaic.sh | ||
[9]: https://github.com/cgewecke/mosaic-1 | ||
[10]: https://github.com/ethereum/web3.js/blob/1.x/scripts/e2e.ganache.core.sh | ||
[11]: https://github.com/trufflesuite/ganache-core | ||
|
||
[1]: https://github.com/ethereum/web3.js/blob/1.x/test/eth.accounts.sign.js | ||
[2]: https://github.com/ethereum/web3.js/blob/1.x/test/e2e.contract.events.js | ||
[3]: https://github.com/ethereum/web3.js/blob/1.x/test/e2e.minified.js | ||
[4]: https://github.com/ethereum/web3.js/blob/1.x/scripts/e2e.cdn.sh | ||
[5]: https://github.com/ethereum/web3.js/blob/1.x/scripts/e2e.windows.sh |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cool, but might be a little confusing since other PRs can overwrite it at any time. just thinking out loud, i wonder if we could "append" to the
index.html
in surge, that would be cool to have a running log of successful (or not) results.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes this would be ideal: e.g
sudden.playground.surge.sh/<branch>/<commit>
As I read surge issue 119, there's no built-in support for this per se and we'd need to maintain our own store of previously published pages.
We can definitely highlight the potential overwrite more clearly in the published page though, will amend this PR for that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
834cb78