E2E Testing Spike

[chatton]

Summary

This issue tracks the work for the E2E Testing investigation.

Problem Definition

We want to be able to easily write automated E2E tests to verify the correctness of code in the ibc-go repo by
creating real instances of simd and icad and a relayer, performing operations and asserting conditions.

Proposal

Investigate existing frameworks or the work involved in writing our own solution.

Tasks

• Create a Dockerfile for simd

• Create a Dockerfile for icad

• Experiment with home grown solution. (After some experimentation, this is definitely re-inventing the wheel, a lot of this work has already been done with this pr and ibctest)

• Missing from ibctest that we would need to use it to write tests:

  • Support for exposing relayer information (required for fee middleware tests)
  • Enable broadcasting of transactions so that we can use go clients to sign transactions in the tests. (And not depend on docker exec to do operations)
  • Arbitrary execution of commands so that we can test features before they are released.
  • Fine grained control over channel creation
  • Allow for the ability to create users using specific mnemonics
  • Configurable ports for the docker containers

• Github action CI work

  • Create github action to run e2e tests under "e2e" directory.
  • Build images from the code of PRs, and push to ghcr registry.
  • Dynamically build test matrix based on tests under the "e2e" directory.

• Write some example tests with the ibctest framwork based on these scenarios

  • TestBasicIBCTransfer
  • TestFeeMiddlewareAsyncSingleSenderTimesOut
  • TestFeeMiddlewareAsyncSingleSender
  • TestFeeMiddlewareAsyncMultipleSenders
  • TestFeeMiddlewareSyncSingleSender depends on Exposing private keys of test users and Configurable ports for the docker containers

The work is being done here

Split into issues

• Add tool to dynamically generate E2E test matrix #1642
• Add E2E Github Actions for E2E Tests #1643
• Add Baseline E2E Test Suite type (E2E #3) #1648
• Add Test Cases
  • E2E: MsgPayPacketFeeAsyncSingleSender #1652
  • E2E: Fee middleware counterparty payee not set #1686
  • E2E: Fee middleware single sender timeout  #1685
  • E2E: Fee middleware multiple senders #1684
  • E2E Fee Middleware Sync Single Sender #1752
• Add E2E README which outlines how to write E2E tests #1683

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged/assigned