[Issue ##2798] Break up e2e tests #2599
Conversation
acouch
force-pushed
the
acouch/shard-e2e-tests
branch
3 times, most recently
from
c20a5fa
to
e68e705
Compare
|
If you want to shave some additional time on the docker builds, have you tried to use npm commands to run tests in the Dockerfile? - can remove the make install and copying of the Makefile over . |
acouch
force-pushed
the
acouch/shard-e2e-tests
branch
2 times, most recently
from
ab1e31c
to
7c33688
Compare
acouch
changed the title
acouch
force-pushed
the
acouch/shard-e2e-tests
branch
from
bf821af
to
b64c9ab
Compare
acouch
requested review from
coilysiren,
andycochran and
doug-s-nava
as code owners
| - name: Run e2e tests (Shard ${{ matrix.shard }}/${{ matrix.total_shards }}) | ||
| env: | ||
| CI: true | ||
| TOTAL_SHARDS: ${{ matrix.total_shards }} |
There was a problem hiding this comment.
curious about the decision to use environment variables, which I can't find a reference to in the docs, versus the --shards flag that seems to be recommended https://playwright.dev/docs/test-sharding#sharding-tests-between-multiple-machines
There was a problem hiding this comment.
This strategy was inherited from the platform, @rylew1 ?
| cat .env.development >> .env.local | ||
| npm run test:e2e | ||
| npm run build |
There was a problem hiding this comment.
it looks like since the sharded test runs are all going in parallel that the npm and playwright setup steps are not able to be cached, so 5 mins of processing time is being spent on each run on something that it seems like could be shared. I'm a novice with github actions, but is this a place where a reusable workflow could help? https://docs.github.com/en/actions/sharing-automations/reusing-workflows#creating-a-reusable-workflow
There was a problem hiding this comment.
Yes, this would be a good follow up.
There was a problem hiding this comment.
I created ticket #2813
Summary
Fixes #2798
Time to review: 5 mins
Changes proposed
Update e2e tests so they are sharded across 3 processes that run in parallel. Time per test reduced by a couple of mins. More importantly the tests seem less flaky. This not as big a deal now but the tests should scale as we grow.