feat: e2e test suite #1253

alexanderbez · 2022-02-10T15:07:09Z

This PR sets up the scaffolding of a full E2E test suite. It includes:

Starting two independent Gaia networks
Connecting them via IBC
Utility functions to send funds over IBC

codecov · 2022-02-10T15:24:18Z

Codecov Report

Merging #1253 (aa18878) into main (066ce7a) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1253   +/-   ##
=======================================
  Coverage   12.78%   12.78%           
=======================================
  Files           9        9           
  Lines        1259     1259           
=======================================
  Hits          161      161           
  Misses       1092     1092           
  Partials        6        6

alexanderbez · 2022-02-11T14:42:45Z

@yaruwangway this is almost ready to test. The two missing pieces are:

Build and push the Hermes docker image
Build and push the Gaia docker image

I can do these, but I would (A) need to know what our docker org is and (B) have access to push to it.

okwme · 2022-02-16T16:17:08Z

@alexanderbez we've been reliant on Tendermint's Docker org til now. Is it possible to use the Github Container Registry for the Cosmos org?

tests/e2e/e2e_setup_test.go

alexanderbez · 2022-02-25T16:23:36Z

@yaruwangway do you think you could help me resolve the CI issues?

alexanderbez · 2022-02-25T17:50:19Z

I need to make one small tweak :)

tokamak-git · 2022-02-28T12:31:24Z

Hey bez, let me know if I can help out with the CI issues...

alexanderbez · 2022-02-28T14:30:41Z

All that needs to happen now is that we need to build and push the hermes Docker file to our dockerhub. Currently, I have it named cosmos/hermes-e2e.

tokamak-git · 2022-02-28T16:03:32Z

Ok sure, Ill pick that up first thing in the morning tomorrow.

yaruwangway · 2022-03-02T10:41:31Z

@yaruwangway do you think you could help me resolve the CI issues?

Hi @alexanderbez , I am not good at this, but I can have a look!

pratikbin · 2022-03-04T04:14:14Z

Can I ask you something
~~- Why are you using debian:buster-slim and coping hermes?~~
~~- Dockerfile name convention SHOULD be Dockerfile or Dockerfile.<name>~~

tokamak-git · 2022-03-07T14:26:29Z

e2e.Dockerfile

+# Compile the gaiad binary
+FROM golang:1.17-alpine AS gaiad-builder
+WORKDIR /src/app/
+COPY go.mod go.sum* ./


hey bez, do you think it would make sense to have a single copy for all the files? just to clean the dockerfile?

It certainly works, but it would be cache inefficient. This is cache efficient.

tokamak-git · 2022-03-07T14:27:33Z

e2e.Dockerfile

+RUN CGO_ENABLED=0 make install
+
+# Add to a distroless container
+FROM gcr.io/distroless/cc:$IMG_TAG


not sure the purpose of the distroless continer? could we use a standard scratch container image here?

We can use scratch actually, but then we have to manually add certificates and all like this

FROM alpine as certs RUN apk add ca-certificates --no-cache ... FROM scratch COPY --from=certs /etc/ssl/certs/ /etc/ssl/certs/ ...

So I would suggest to use gcr.io/distroless/static since we are CGO_ENABLED=0

tokamak-git · 2022-03-07T14:34:27Z

Makefile


-benchmark:
-	@go test -mod=readonly -bench=. ./...
+docker-build-debug:


hey, whats the purpose of this docker build?

tokamak-git · 2022-03-07T15:11:44Z

tests/e2e/chain.go

+)
+
+const (
+	keyringPassphrase = "testpassphrase"


shall we configure injeccting these via env vars? could help in automation of testing down the line for different senarios. what do you think?

tokamak-git · 2022-03-07T15:16:21Z

tests/e2e/docker/hermes.Dockerfile

@@ -0,0 +1,12 @@
+FROM informalsystems/hermes:0.12.0 AS hermes-builder


guess it would make sense to pass the hermies version via a config/env just so that we can use different published images. wdyt?

tokamak-git · 2022-03-07T15:26:53Z

tests/e2e/e2e_test.go

+	sdk "github.com/cosmos/cosmos-sdk/types"
+)
+
+func (s *IntegrationTestSuite) TestIBCTokenTransfer() {


Shall I add a description so as to add context for noobes?

tokamak-git · 2022-03-07T15:28:38Z

tests/e2e/e2e_util_test.go

+}
+
+func queryGaiaTx(endpoint, txHash string) error {
+	resp, err := http.Get(fmt.Sprintf("%s/cosmos/tx/v1beta1/txs/%s", endpoint, txHash))


A question really, should we interact with the chains over somthing other than rest?
essentially seeing as not all types of query/commands are available via rest, maybe rpc/grpc is a better approach in this case?

tokamak-git · 2022-03-07T15:29:44Z

tests/e2e/genesis.go

+	genFile := config.GenesisFile()
+	doc := &tmtypes.GenesisDoc{}
+
+	if _, err := os.Stat(genFile); err != nil {


should we modify this to use ioutils.ReadAll just so as to reduce a few lines of code?

tokamak-git · 2022-03-07T15:32:23Z

tests/e2e/scripts/hermes_bootstrap.sh

+touch /root/.hermes/config.toml
+
+# setup Hermes relayer configuration
+tee /root/.hermes/config.toml <<EOF


any thoughts of making copying this as a file from our repository? like a test/hermies_config.toml for example?

tokamak-git · 2022-03-07T15:34:35Z

tests/e2e/validator.go

+	gaia "github.com/cosmos/gaia/v6/app"
+)
+
+type validator struct {


Im not sure, but do you think we can reuse the validator type from the sdk?

tokamak-git

LGTM

tokamak-git · 2022-03-09T14:08:55Z

@alexanderbez checked the link that is reported as broken, looks to be working now. Guess all we need to do is to rerun the actions associated with the pr.

okwme · 2022-03-09T22:52:52Z

@tokamak-git started re-running the link checker job. @alexanderbez your final "build and push the hermes docker", this is completed?

okwme · 2022-03-09T23:28:13Z

looks like we're having some docker issues @tokamak-git :

e2e_setup_test.go:56
        	Error:      	Received unexpected error:
        	            	API error (404): pull access denied for cosmos/hermes-e2e, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

i dont' see anything here: https://github.com/cosmos/hermes-e2e
is that where it's supposed to be now?

alexanderbez · 2022-03-11T20:46:05Z

I had to make the hermes-e2e package public. Rerunning the test suite....

tests/e2e/e2e_setup_test.go

alexanderbez · 2022-03-11T20:53:19Z

Also had to fix the repo name to ghcr.io/cosmos/hermes-e2e.

alexanderbez added 7 commits February 10, 2022 10:06

init commit

8261f95

updates

54b2aa9

updates

5d5d256

updates

ee9b5f4

updates

8882240

updates

b1ed4ea

updates

abdb62e

alexanderbez and others added 2 commits February 11, 2022 09:40

updates

db02af3

Merge branch 'main' into bez/e2e-test-setup

d38f4b7

yaruwangway assigned tokamak-git Feb 16, 2022

Merge branch 'main' into bez/e2e-test-setup

eefc8bc

okwme reviewed Feb 23, 2022

View reviewed changes

tests/e2e/e2e_setup_test.go Outdated Show resolved Hide resolved

alexanderbez added 8 commits February 23, 2022 12:03

updates

0b08f58

updates

3b694f5

updates

bd4e604

updates

9ff2948

updates

a79b816

updates

c31be38

ci++

2500e43

ci++

5f47f16

okwme added 6 commits February 25, 2022 17:32

tweak sims.yml to use go 1.17

d8728bf

Update release-sims.yml

b7c3328

Update sims.yml

a2c6900

sdk compatibility

a32dc7a

sdk compatibility

b755562

bump action to go v1.17

c8faeb7

alexanderbez requested review from yaruwangway, jgimeno, cmwaters, jackzampolin, mmulji-ic, glnro and tokamak-git as code owners February 25, 2022 17:34

mmulji-ic mentioned this pull request Feb 28, 2022

Identify and develop gaia-specific tests #846

Closed

6 tasks

yaruwangway mentioned this pull request Mar 2, 2022

EPIC: tests - compatibility, e2e, github action #1318

Closed

4 tasks

Merge branch 'main' into bez/e2e-test-setup

6157665

okwme requested a review from crodriguezvega as a code owner March 4, 2022 15:58

Update validator-setup.md

1fcc72e

tokamak-git reviewed Mar 7, 2022

View reviewed changes

tokamak-git approved these changes Mar 9, 2022

View reviewed changes

alexanderbez commented Mar 11, 2022

View reviewed changes

tests/e2e/e2e_setup_test.go Outdated Show resolved Hide resolved

Update tests/e2e/e2e_setup_test.go

aa18878

alexanderbez merged commit d6f7b57 into main Mar 11, 2022

alexanderbez deleted the bez/e2e-test-setup branch March 11, 2022 21:09

chatton mentioned this pull request Jun 27, 2022

E2E Testing Spike cosmos/ibc-go#1583

Closed

33 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: e2e test suite #1253

feat: e2e test suite #1253

alexanderbez commented Feb 10, 2022

codecov bot commented Feb 10, 2022 •

edited

Loading

alexanderbez commented Feb 11, 2022 •

edited

Loading

okwme commented Feb 16, 2022

alexanderbez commented Feb 25, 2022

alexanderbez commented Feb 25, 2022

tokamak-git commented Feb 28, 2022

alexanderbez commented Feb 28, 2022

tokamak-git commented Feb 28, 2022

yaruwangway commented Mar 2, 2022

pratikbin commented Mar 4, 2022 •

edited

Loading

tokamak-git Mar 7, 2022

pratikbin Mar 7, 2022

tokamak-git Mar 7, 2022

pratikbin Mar 7, 2022

tokamak-git Mar 7, 2022

tokamak-git Mar 7, 2022

tokamak-git Mar 7, 2022

tokamak-git Mar 7, 2022

tokamak-git Mar 7, 2022

tokamak-git Mar 7, 2022

tokamak-git Mar 7, 2022

tokamak-git Mar 7, 2022

tokamak-git left a comment

tokamak-git commented Mar 9, 2022

okwme commented Mar 9, 2022

okwme commented Mar 9, 2022

alexanderbez commented Mar 11, 2022

alexanderbez commented Mar 11, 2022

		@@ -0,0 +1,12 @@
		FROM informalsystems/hermes:0.12.0 AS hermes-builder

feat: e2e test suite #1253

feat: e2e test suite #1253

Conversation

alexanderbez commented Feb 10, 2022

codecov bot commented Feb 10, 2022 • edited Loading

Codecov Report

alexanderbez commented Feb 11, 2022 • edited Loading

okwme commented Feb 16, 2022

alexanderbez commented Feb 25, 2022

alexanderbez commented Feb 25, 2022

tokamak-git commented Feb 28, 2022

alexanderbez commented Feb 28, 2022

tokamak-git commented Feb 28, 2022

yaruwangway commented Mar 2, 2022

pratikbin commented Mar 4, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tokamak-git left a comment

Choose a reason for hiding this comment

tokamak-git commented Mar 9, 2022

okwme commented Mar 9, 2022

okwme commented Mar 9, 2022

alexanderbez commented Mar 11, 2022

alexanderbez commented Mar 11, 2022

codecov bot commented Feb 10, 2022 •

edited

Loading

alexanderbez commented Feb 11, 2022 •

edited

Loading

pratikbin commented Mar 4, 2022 •

edited

Loading