Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Colima #531

Closed
wants to merge 36 commits into from
Closed

Add support for Colima #531

wants to merge 36 commits into from

Conversation

cristianrgreco
Copy link
Collaborator

@cristianrgreco cristianrgreco commented Apr 13, 2023

The goal of this PR was to integrate support for Colima into testcontainers-node such that we could run the entire test suite with Colima as part of our CICD. As of 19/04/2023 this isn't possible for the following reasons:

  1. Runner performance: Colima can only run on MacOS GHA runners, as only these runners support nested virtualisation. During this PR, the performance of the MacOS GHA runners has been terrible, with test runs ranging from 25m-55m, whereas with Docker and Podman we get consistent runs of ~13m. This is likely related to this issue which has been closed by GH: Performance of GitHub-hosted macOS runner actions/runner-images#1336. Although GH very recently added support for nested virtualisation in their Linux runners, this is hidden behind GH enterprise, and behind their larger runners which require payment. I therefore cannot rule out, that the following issues are in some way related to the MacOS runners, instead of Colima.
  2. Flakiness:
    2.1. We intermittently see Docker pull rate limits for official images, such as Mongo, PostgreSQL, etc (?!).
    2.2. Dockerfiles which install dependencies when built are randomly unable to resolve dependencies, such as curl.
    2.3. I have had to increase timeouts of some test suites by a multiplier, so that the tests complete in time.
  3. Colima port forwarding delays: Because of the way Colima works, a container port can be bound, but inaccessible for seconds until it is port forwarded. We therefore run into the strange case where a container can bind a port, write a log message to say that it's ready, and yet when we try to connect we get connection refused. This PR attempted to work around it by changing all wait strategies to be a composite of its existing wait strategy + port wait strategy. This has polluted the codebase with a very specific Colima issue, to the extent where tests for other wait strategies have to include the port wait strategy: Connection refused right after container start abiosoft/colima#71.
  4. Colima performance: The test suite cannot be run in parallel using Colima, unlike with other container runtimes such as Docker and Podman. Possibly related to Colima still crashing under network load abiosoft/colima#603.
  5. Lack of IPv6 support: Node versions >= 18 changed the way they resolve DNS, to respect the OS order. Therefore Node 18 and greater, in an IPv6 enabled environment, will resolve localhost to its IPv6 equivalent. Colima does not support IPv6, and so this has to be explicitly disabled: IPv6 localhost not supported abiosoft/colima#583

To re-emphasise, because of the way GH have structured its runners, I cannot rule out that a lot of the issues I've faced are because of the MacOS runners.

This comment is not to say that Colima is unsupported, it means that your mileage may vary. I will soon document some of these issues and workarounds in the official docs, and depending on your use cases and environment, Colima may work without issue.

I will re-visit Colima support in the future. In the meantime if I have missed anything, or someone is able to improve Colima support in testcontainers-node, any contributions are very welcome!

@netlify
Copy link

netlify bot commented Apr 13, 2023

Deploy Preview for testcontainers-node ready!

Name Link
🔨 Latest commit 7074f7e
🔍 Latest deploy log https://app.netlify.com/sites/testcontainers-node/deploys/643e4d2039303d0008b76183
😎 Deploy Preview https://deploy-preview-531--testcontainers-node.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

@cristianrgreco cristianrgreco added enhancement New feature or request minor Backward compatible functionality labels Apr 13, 2023
@cristianrgreco cristianrgreco self-assigned this Apr 17, 2023
@glasser
Copy link

glasser commented Apr 19, 2023

I've been looking into what it would take to get this working for Java as well, though it sounds like @eddumelendez might take care of that once this is done. Just out of curiosity, is the issue that waiting for log lines just isn't good enough for Colima, and you have to use a port-based wait strategy instead in practice?

@cristianrgreco
Copy link
Collaborator Author

cristianrgreco commented Apr 19, 2023

Hi @glasser, I've just updated the PR comment with all the info

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request minor Backward compatible functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants