Script to run full set of backwards compatibility tests #2047
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…et-of-backwards-compatibility-tests
…et-of-backwards-compatibility-tests
…et-of-backwards-compatibility-tests
…ards-compatibility-tests' of https://github.com/cosmos/ibc-go into cian/issue#2029-add-script-to-run-the-full-set-of-backwards-compatibility-tests
chatton
requested review from
colin-axner,
AdityaSripal,
crodriguezvega,
seantking,
charleenfei and
damiannolan
as code owners
chatton
commented
Aug 19, 2022
| @@ -25,7 +25,7 @@ on: | |||
| - master | |||
| - v0.3.2 | |||
| - v0.2.2 | |||
| - v0.1.2 | |||
| - v0.1.3 | |||
There was a problem hiding this comment.
tests should pass with this version once this pr is merged
crodriguezvega
approved these changes
Aug 19, 2022
There was a problem hiding this comment.
This is absolute wizardry, @chatton! 🧙♂️
Is it possible to have also a workflow to run this tests from GitHub's UI?
|
@crodriguezvega I'm not actually sure if it's possible to trigger a workflow from inside a workflow 🤔 , but I don't see why it shouldn't work, I can look into it this! I left it relatively simple for now, the markdown format output let's us do whatever we want with it, and we can always include the output in other workflows or processes. |
Co-authored-by: Carlos Rodriguez <carlos@interchain.io>
…et-of-backwards-compatibility-tests
chatton
deleted the
cian/issue#2029-add-script-to-run-the-full-set-of-backwards-compatibility-tests
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR introduces a script which reads in a json configuration of tests to run and create a Github Workflow with the specified values.
The script outputs markdown, this makes it easy to just copy paste the output into a github issue to track test results. And also ensures it is easily extendable if we want to automate the creation of github issues using the contents.
Each time we want to run tests against a different set of versions, we just need to make one of these configurations. They will likely be largely similar configurations, with different versions specified. They live under
scripts/test-matriciesThe script assumes you have both
jqandghinstalled.The below is output from running
make compatibility-testsRaw:
Rendered:
Backwards compatibility tests
Matrix Version: main
TestTransferTestSuite
TestFeeMiddlewareTestSuite
TestInterchainAccountsTestSuite
closes: #2029
Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.
docs/) or specification (x/<module>/spec/)godoccomments.Unreleasedsection inCHANGELOG.mdFiles changedin the Github PR explorerCodecov Reportin the comment section below once CI passes