Docker compose: add configuration setting for selecting running services to bind

[dsyer]

I have a docker-compose.yml that starts multiple services but I only want to bind to one (or some) of them. It is possible to do that currently by re-arranging the compose file into "profiles" and setting spring.docker.compose.profiles.active. But I might not be able to do that for an arbitrary set of services (and I might not want or be able to edit the compose file either), so it would be nice to be able to simply list (or pattern match) the services to bind to.

[wilkinsona]

You can ignore services by labelling them with org.springframework.boot.ignore. That will require editing your docker-compose.yml file. Why might you not be able to do that?

[dsyer]

I didn’t know about that, and it might be useful, but not in general. Imagine I want to switch between 2 services for independent tests, for example - it would be awkward to have to edit the compose config in between test invocations.

[wilkinsona]

The docker compose support isn't aimed at tests. It's intended for use when running the whole app during development as a replacement for manually running docker compose up and then configuring properties to connect to the resulting service.

I'm not opposed to adding something but I don't think there's a compelling use case yet.

[spring-projects-issues]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

[dsyer]

I don't know why the use case I mentioned isn't enough to at least keep the issue open.

[philwebb]

I was thinking about this one the other day and I wonder if we could offer a way to apply labels the application.yaml. Something like:

spring:
  docker:
    compose:
      labels:
        my-service:
          org.springframework.boot.ignore: true

[dsyer]

That would work, but why does it work better than simply listing the services you want to bind? To me it makes sense that I am mirroring the docker-compose command line (like with profiles). "I tell you where to find the YAML and what arguments to add to docker-compose, you run it for me." Labels seem like an unnecessary (in this case) indirection.

[philwebb]

The main reason behind my thinking was we could support the other labels (such as org.springframework.boot.readiness-check.tcp.disable), but listing services is probably easier to follow.

[mroche89]

Big +1 to this

That will require editing your docker-compose.yml file. Why might you not be able to do that?

In my case, we have a few different repos that each define their own compose file for how to run that service.

Our compose file for ServiceA has something like

include:
    - ../serviceB/docker-compose.yml

...

This means that to run this particular compose file I'd need to add ignore labels to the other repos compose file, which I don't want because it would mean the other service won't create service connections.

Specifying which containers to bind to will also mean changes to service names or the addition/removal of services in the included compose does not affect my settings unless I explicitly bind to external containers

[aldex32]

I do like what @dsyer is proposing

[erihanse]

We have a use-case. I understand it's not meant for testing. However, we have in our dockerfile how to run the app being built for the same project. Also we have the postgres db.

services:
  the-app:
    container_name: the-app
    image: the-app
    build:
      context: .
    ports:
      - '8082:8082'

  postgres:
    container_name: postgres
    image: postgres
    ports:
      - '5432:5432'

When running in gradle/intellij directly, or when running integration tests, it would useful to be able to specify only the db should be spinned up. However, in other scenarios we'd also like to use docker-compose to spin up both, through docker compose (not through spring). I think we can work around this by using docker compose profiles. But we have a convention across our many projects that running docker compose up from cli should mean the whole stack for that project, so we would have to change our conventions to make it work with spring boot docker compose.

[nosan]

@philwebb @wilkinsona
I started working on this issue a little bit and just want to get feedback from you on whether I am in the right direction or not.

main...nosan:spring-boot:35059

Or maybe passing services as an argument to Docker Compose is better?

void up(LogLevel logLevel, List<String> arguments, Set<String> services);

[wilkinsona]

With Boot 3.4, this can be achieved using the general support for passing arguments to the "start" command (docker compose up or docker compose start). Thus far, we've preferred to use the general arguments rather than adding properties for specific options such as --project-name.

Consider this compose file:

services:
  mysql:
    image: 'mysql:latest'
    environment:
      - 'MYSQL_DATABASE=mydatabase'
      - 'MYSQL_PASSWORD=secret'
      - 'MYSQL_ROOT_PASSWORD=verysecret'
      - 'MYSQL_USER=myuser'
    ports:
      - '3306'
  postgres:
    image: 'postgres:latest'
    environment:
      - 'POSTGRES_DB=mydatabase'
      - 'POSTGRES_PASSWORD=secret'
      - 'POSTGRES_USER=myuser'
    ports:
      - '5432'

Starting the application normally will launch containers for both MySQL and Postgres:

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/

 :: Spring Boot ::       (v3.4.0-SNAPSHOT)

2024-10-14T10:45:50.284+01:00  INFO 17277 --- [gh-35059] [           main] c.example.gh_35059.Gh35059Application    : Starting Gh35059Application using Java 17.0.12 with PID 17277 (/Users/awilkinson/dev/temp/gh-35059/build/classes/java/main started by awilkinson in /Users/awilkinson/dev/temp/gh-35059)
2024-10-14T10:45:50.287+01:00  INFO 17277 --- [gh-35059] [           main] c.example.gh_35059.Gh35059Application    : No active profile set, falling back to 1 default profile: "default"
2024-10-14T10:45:50.341+01:00  INFO 17277 --- [gh-35059] [           main] .s.b.d.c.l.DockerComposeLifecycleManager : Using Docker Compose file /Users/awilkinson/dev/temp/gh-35059/compose.yaml
2024-10-14T10:45:51.992+01:00  INFO 17277 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-mysql-1  Created
2024-10-14T10:45:51.992+01:00  INFO 17277 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-postgres-1  Created
2024-10-14T10:45:51.998+01:00  INFO 17277 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-postgres-1  Starting
2024-10-14T10:45:51.999+01:00  INFO 17277 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-mysql-1  Starting
2024-10-14T10:45:52.500+01:00  INFO 17277 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-postgres-1  Started
2024-10-14T10:45:52.545+01:00  INFO 17277 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-mysql-1  Started
2024-10-14T10:45:52.545+01:00  INFO 17277 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-postgres-1  Waiting
2024-10-14T10:45:52.546+01:00  INFO 17277 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-mysql-1  Waiting
2024-10-14T10:45:53.054+01:00  INFO 17277 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-postgres-1  Healthy
2024-10-14T10:45:53.055+01:00  INFO 17277 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-mysql-1  Healthy
2024-10-14T10:45:55.494+01:00  INFO 17277 --- [gh-35059] [           main] c.example.gh_35059.Gh35059Application    : Started Gh35059Application in 5.608 seconds (process running for 6.074)

To start only mysql, spring.docker.compose.start.arguments can be set to mysql and only mysql will be started:

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/

 :: Spring Boot ::       (v3.4.0-SNAPSHOT)

2024-10-14T10:46:50.287+01:00  INFO 17489 --- [gh-35059] [           main] c.example.gh_35059.Gh35059Application    : Starting Gh35059Application using Java 17.0.12 with PID 17489 (/Users/awilkinson/dev/temp/gh-35059/build/classes/java/main started by awilkinson in /Users/awilkinson/dev/temp/gh-35059)
2024-10-14T10:46:50.289+01:00  INFO 17489 --- [gh-35059] [           main] c.example.gh_35059.Gh35059Application    : No active profile set, falling back to 1 default profile: "default"
2024-10-14T10:46:50.341+01:00  INFO 17489 --- [gh-35059] [           main] .s.b.d.c.l.DockerComposeLifecycleManager : Using Docker Compose file /Users/awilkinson/dev/temp/gh-35059/compose.yaml
2024-10-14T10:46:51.667+01:00  INFO 17489 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-mysql-1  Created
2024-10-14T10:46:51.674+01:00  INFO 17489 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-mysql-1  Starting
2024-10-14T10:46:51.982+01:00  INFO 17489 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-mysql-1  Started
2024-10-14T10:46:51.982+01:00  INFO 17489 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-mysql-1  Waiting
2024-10-14T10:46:52.493+01:00  INFO 17489 --- [gh-35059] [utReader-stderr] o.s.boot.docker.compose.core.DockerCli   :  Container gh-35059-mysql-1  Healthy
2024-10-14T10:46:54.308+01:00  INFO 17489 --- [gh-35059] [           main] c.example.gh_35059.Gh35059Application    : Started Gh35059Application in 4.367 seconds (process running for 4.787)

The arguments are a list so you can specify multiple services using a comma-separated value of YAML's list syntax as needed.

@mroche89 @dsyer please give this a try with Boot 3.4 and let us know if it meets your needs. I used 3.4.0-SNAPSHOT but the feature's available in 3.4.0-M3 if you'd prefer something that isn't a moving target. @mroche89, milestones are available from https://repo.spring.io/milestone (and snapshots from https://repo.spring.io/snapshot).

[dsyer]

I think spring.docker.compose.start.arguments works for options that apply on the command line after docker compose up. Are they also applied to docker compose down (they might not be legal there)? I suppose that also rules out options to docker compose itself (https://docs.docker.com/reference/cli/docker/compose/), as opposed to the sub-command, but probably that's not much of a problem since several of them are already supported by Spring Boot in other ways (e.g. specifying a docker-compose.yml location). Anyway, it works for me in the PetClinic, e.g. in relation to spring-projects/spring-petclinic#1522.

[wilkinsona]

Thanks for trying it, Dave.

Are they also applied to docker compose down (they might not be legal there)?

No, we have spring.docker.compose.stop.arguments for docker compose down and docker compose stop.

I suppose that also rules out options to docker compose itself

You can use spring.docker.compose.arguments for arguments to pass to docker compose (only in snapshots at the moment). That's how we're supporting --project-name for example.

I'll leave this open to see how @mroche89 (or anyone else who was interested in this feature) fares, but it's looking like we're good here and will be able to close this one as having been superseded by #42571 and #38763

[spring-projects-issues]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

[spring-projects-issues]

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.