Add WaitFor::Healthy

[andrey-yantsen]

Currently implemented waiting conditions can't guarantee that the application in the container is ready and is working correctly.

Some of the containers have healthchecks (e.g. plexinc/pms-docker) and for those it could be useful to rely on the healthcheck results instead of processing stdout/stderr.

[jimmiebfulton]

Looking at the Java Testcontainers, there is a facility for performing HTTP checks, which provides a generalized strategy for something like WaitFor::HttpStatus:

    public CockroachContainer(final DockerImageName dockerImageName) {
        super(dockerImageName);

        dockerImageName.assertCompatibleWith(DEFAULT_IMAGE_NAME);

        withExposedPorts(REST_API_PORT, DB_PORT);
        waitingFor(
            new HttpWaitStrategy()
                .forPath("/health")
                .forPort(REST_API_PORT)
                .forStatusCode(200)
                .withStartupTimeout(Duration.ofMinutes(1))
        );
        withCommand("start-single-node --insecure");
    }

[thomaseizinger]

I'd merge a PR that adds WaitFor::HttpStatus and a constructor function for conveniently creating an instance of that.

[andrey-yantsen]

Having WaitFor::HttpStatus is a good thing, but I don't think it could be considered a replacement for the WaitFor::Healthy. In the docker-library/healthcheck, you can see some sample health-checks for the official docker containers. For example, here's the one for rabbitmq — it just executes a command inside the container and decides whether it's healthy based on the results.

[thomaseizinger]

All of WaitFor is about determining the health of the container so WaitFor::Healthy doesn't make much sense by itself.

Instead, we need to identify the various strategies for determining health (log message, port open, http status, etc) and add those as variants to WaitFor.

A particular image can then compose those strategies together for an overall "wait until healthy" behaviour :)

I haven't looked at the rabbitmq thing in detail but executing a command and checking its return value might be another viable WaitFor variant although one could argue that the image should just run said command during startup and you wait for the log message instead.

[andrey-yantsen]

@thomaseizinger the Java testcontainers has the DockerHealthcheckWaitStrategy (docs) though :)

[andrey-yantsen]

Probably I was wrong suggesting the name Healthy :) In any case — I want the same functionality that DockerHealthcheckWaitStrategy provides. I.e. waiting for the container status to be healthy in Docker.

[thomaseizinger]

@thomaseizinger the Java testcontainers has the DockerHealthcheckWaitStrategy (docs) though :)

I didn't know docker had this feature, interesting!

Is there a CLI flag that we can use to wait for the health checks before the startup is considered complete?

[thomaseizinger]

Probably I was wrong suggesting the name Healthy :) In any case — I want the same functionality that DockerHealthcheckWaitStrategy provides. I.e. waiting for the container status to be healthy in Docker.

After reading these docs a bit, WaitFor::Healthcheck certainly makes sense!

[andrey-yantsen]

Glad we're on the same page now :)

Is there a CLI flag that we can use to wait for the health checks before the startup is considered complete?

I'm using the following command to detect the container's status:

docker inspect --format "{{ .State.Health.Status }}" <container>

It returns either healthy or unhealthy (maybe some other values too, but I think we're interested only in healthy here).

[thomaseizinger]

I'm using the following command to detect the container's status:\n\ndocker inspect --format "{{ .State.Health.Status }}" \u003Ccontainer>\nIt returns either healthy or unhealthy (maybe some other values too, but I think we're interested only in healthy here).

Thanks for that. It is unfortunate that the CLI doesn't allow you to pass --require-healthy or something during startup.

Meaning we need to implement a busy waiting loop ourselves.

[andrey-yantsen]

Well, the first step is done, now we need to wait for a while :) softprops/shiplift#323

[andrey-yantsen]

Or, perhaps, it could be better to switch to another crate :\ softprops/shiplift#321