Skip to content

Stop worrying about manually crafting a parallel CI pipeline and start folding execution time by forming wormholes in your CI workflows ๐Ÿš€

Notifications You must be signed in to change notification settings

skorfmann/wormhole

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

4 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Wormhole CI

Stop worrying about manually crafting a parallel CI pipeline and start folding execution time by forming wormholes in your CI workflows ๐Ÿš€

wormhole

Components

How does it work

Jest uses workers to run parallel tests. It's essentially possible to run each test file on its own worker. That's exactly what we're doing here. However, rather than running the test locally, the worker calls out to Lambda function and reports the result back to the Jest process.

workflow

Status

This is a proof of concept at this point. The goal is to drastically reduce the run time of integration test pipelines without sacrificing developer experience.

Things to figure out

  • Reduce deployment time (biggest gain is likely correlated with smaller Docker images)
  • Local EFS mount, rather than code deployments for local testing (needs VPN or a proxy)
  • Alternatively, use a remote container for VS code which could easily mount EFS
  • Handle test specific settings / runners (e.g. a specific Docker image for a test / test group)
  • Is there another way of splitting tests than files? (patterns maybe)

Alternative runners

It would be great to support more runner types. This would likely enable Windows and Mac builds as well.

  • Use Fargate as runner
  • Use Google Cloud run as runner
  • Use Azure Functions as runner
  • Use Azure App Service as runner

Usage

Make sure to have valid AWS credentials in your env.

yarn install
yarn build
cd packages/lambda
cdktf deploy --auto-approve
cd ../../examples/terraform
yarn test

About

Stop worrying about manually crafting a parallel CI pipeline and start folding execution time by forming wormholes in your CI workflows ๐Ÿš€

Topics

Resources

Stars

Watchers

Forks