Starting Elasticsearch in Gitlab-CI fails

[arian]

Describe the bug

I try to use testcontainers for Elasticsearch in Gitlab-CI, but it fails when trying to connect to Elasticsearch.

To Reproduce

test:
  image: python:3.11
  variables:
    DOCKER_HOST: "tcp://docker:2375"
    DOCKER_TLS_CERTDIR: ""
    DOCKER_DRIVER: overlay2
  services:
    - name: docker:dind
      command: ["--tls=false"]

  before_script:
    - python -V
    - pip install testcontainers
  script:
    - python test/test.py

from testcontainers.elasticsearch import ElasticSearchContainer
import logging

logging.basicConfig(level=logging.DEBUG)

with ElasticSearchContainer(f"elasticsearch:8.11.1") as es:
    print(es.get_url())

gitlab-ci logs

$ python test/test.py
DEBUG:docker.utils.config:Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
DEBUG:docker.utils.config:No config file found
DEBUG:docker.utils.config:Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
DEBUG:docker.utils.config:No config file found
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): docker:2375
DEBUG:urllib3.connectionpool:http://docker:2375/ "GET /version HTTP/1.1" 200 806
Pulling image elasticsearch:8.11.1
INFO:testcontainers.core.container:Pulling image elasticsearch:8.11.1
DEBUG:urllib3.connectionpool:http://docker:2375/ "POST /v1.43/containers/create HTTP/1.1" 404 50
DEBUG:docker.auth:Looking for auth config
DEBUG:docker.auth:No auth config in memory - loading from filesystem
DEBUG:docker.utils.config:Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
DEBUG:docker.utils.config:No config file found
DEBUG:docker.auth:Looking for auth entry for 'docker.io'
DEBUG:docker.auth:No entry found
DEBUG:docker.auth:No auth config found
DEBUG:urllib3.connectionpool:http://docker:2375/ "POST /v1.43/images/create?tag=8.11.1&fromImage=elasticsearch HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:http://docker:2375/ "GET /v1.43/images/elasticsearch:8.11.1/json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:http://docker:2375/ "POST /v1.43/containers/create HTTP/1.1" 201 88
DEBUG:urllib3.connectionpool:http://docker:2375/ "GET /v1.43/containers/ce60f2427a2f39168b711d5383c3f4fb96bf597751256021c8e7bc8c0a7d9103/json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:http://docker:2375/ "POST /v1.43/containers/ce60f2427a2f39168b711d5383c3f4fb96bf597751256021c8e7bc8c0a7d9103/start HTTP/1.1" 204 0
Container started: ce60f2427a2f
INFO:testcontainers.core.container:Container started: ce60f2427a2f
Waiting to be ready...
INFO:testcontainers.core.waiting_utils:Waiting to be ready...
Waiting to be ready...
INFO:testcontainers.core.waiting_utils:Waiting to be ready...
DEBUG:urllib3.connectionpool:http://docker:2375/ "GET /v1.43/containers/ce60f2427a2f39168b711d5383c3f4fb96bf597751256021c8e7bc8c0a7d9103/json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:http://docker:2375/ "GET /v1.43/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22id%22%3A+%5B%22ce60f2427a2f39168b711d5383c3f4fb96bf597751256021c8e7bc8c0a7d9103%22%5D%7D HTTP/1.1" 200 None
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/urllib/request.py", line 1348, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/local/lib/python3.11/http/client.py", line 1294, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1340, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1289, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1048, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.11/http/client.py", line 986, in send
    self.connect()
  File "/usr/local/lib/python3.11/http/client.py", line 952, in connect
    self.sock = self._create_connection(
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/socket.py", line 851, in create_connection
    raise exceptions[0]
  File "/usr/local/lib/python3.11/socket.py", line 836, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/builds/arian/project/test/test.py", line 6, in <module>
    with ElasticSearchContainer(f"elasticsearch:8.11.1") as es:
  File "/usr/local/lib/python3.11/site-packages/testcontainers/core/container.py", line 71, in __enter__
    return self.start()
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/testcontainers/elasticsearch.py", line 101, in start
    self._connect()
  File "/usr/local/lib/python3.11/site-packages/testcontainers/core/waiting_utils.py", line 49, in wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/testcontainers/elasticsearch.py", line 90, in _connect
    res = urllib.request.urlopen(self.get_url())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/urllib/request.py", line 519, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/local/lib/python3.11/urllib/request.py", line 1377, in http_open
    return self.do_open(http.client.HTTPConnection, req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 111] Connection refused>
DEBUG:urllib3.connectionpool:Starting new HTTP connection (2): docker:2375
DEBUG:urllib3.connectionpool:http://docker:2375/ "DELETE /v1.43/containers/ce60f2427a2f39168b711d5383c3f4fb96bf597751256021c8e7bc8c0a7d9103?v=True&link=False&force=True HTTP/1.1" 204 0

Solution?

It seems to be that the _connect method throws an ConnectionRefusedError or URLError which isn't handled, which it probably should.

    @wait_container_is_ready()
    def _connect(self):
        res = urllib.request.urlopen(self.get_url())
        if res.status != 200:
            raise Exception()

[arian]

A version that seems to work:

from testcontainers.elasticsearch import ElasticSearchContainer
from urllib.error import URLError
import logging
from testcontainers.core.waiting_utils import wait_container_is_ready

logging.basicConfig(level=logging.DEBUG)

class FixedElasticSearchContainer(ElasticSearchContainer):

    @wait_container_is_ready(URLError)
    def _connect(self):
        return super()._connect()

with FixedElasticSearchContainer(f"elasticsearch:8.11.1") as es:
    print(es.get_url())

[totallyzen]

hey @arian thanks for the triage for us!

Sorry for the delays, but I'm focusing on #405 for the time being and did not establish the habit of triaging so folks are getting some delayed responses.

From what you've said, it seems that you could just contribute the fix if you want. The fix seems simple enough!

Contributions welcome!

[arian]

It seems it's fixed already actually, but not yet released: 0f9ad24

[totallyzen]

@arian since it's already fixed on main but unreleased, I'm closing this issue and would like to redirect everyone to #418 for their comments on when we can release the fixes properly!

Sorry for the delays, but still just squeezing this in our free time and operating with some restrictions (can't just change everything on the repo all at once)

For further questions, please ask on #418, thanks! 🙂