Docker compose is incompatible with the new Docker engine #7162

SillyCode · 2023-02-21T09:08:14Z

Description

Following the upgrade to the Docker and Docker Compose and issue #7080.

Currently the latest windows-2019 release is 20230214.1.

Which contains the following Docker related versions:
Docker 23.0.1 and Docker Compose V2 2.16.0

Ref:
https://github.com/actions/runner-images/releases/tag/win19%2F20230214.1

Using this Windows-2019 image.
We face the issue below:
The current Compose file version is not compatible with your engine version. Please upgrade your Compose file to a more recent version, or set a COMPOSE_API_VERSION in your environment.

My docker-compose file

version: "3.9"
services:
  sql-server-db:
    container_name: my-mssql
    image: my-mssql-image:latest
    ports:
      - "1433:1433"
    environment:
      SA_PASSWORD: "***************"
      ACCEPT_EULA: "Y"

Operating System
Microsoft Windows Server 2019
10.0.17763
Runner Image
Image: windows-2019
Version: 20230214.1

Looking at the build log. We can see that the issue is reported actually by Choco Docker Compose.

2023-02-19T08:22:09.0368683Z ##[debug]which 'docker-compose'
2023-02-19T08:22:09.0752496Z ##[debug]found: 'C:\ProgramData\Chocolatey\bin\docker-compose.exe'
2023-02-19T08:22:10.3052706Z [command]C:\ProgramData\Chocolatey\bin\docker-compose.exe -f D:\a\1\s\Sources\Tests\Docker\docker-compose.yml -f C:\agents\2.217.2\.docker-compose.1676794930282.yml -p SI up -d
2023-02-19T08:22:11.4679494Z Creating network "si_default" with the default driver
2023-02-19T08:22:12.0324536Z Pulling sql-server-db (***/my-mssql-image:latest)...
2023-02-19T08:22:14.8209567Z I2: Pulling from [i2-si-mssql](my-mssql-image:latest)
2023-02-19T08:32:54.0137344Z The current Compose file version is not compatible with your engine version. Please upgrade your Compose file to a more recent version, or set a COMPOSE_API_VERSION in your environment.
2023-02-19T08:32:54.9175774Z ##[debug]Exit code 1 received from tool 'C:\ProgramData\Chocolatey\bin\docker-compose.exe'
2023-02-19T08:32:54.9177252Z ##[debug]STDIO streams have closed for tool 'C:\ProgramData\Chocolatey\bin\docker-compose.exe'
2023-02-19T08:32:55.0020263Z ##[error]Pulling sql-server-db (***/my-mssql-image:latest)...
2023-02-19T08:32:55.0021455Z ##[debug]Processed: ##vso[task.issue type=error;]Pulling sql-server-db (***/my-mssql-image:latest)...
2023-02-19T08:32:55.0023243Z ##[error]The current Compose file version is not compatible with your engine version. Please upgrade your Compose file to a more recent version, or set a COMPOSE_API_VERSION in your environment.
2023-02-19T08:32:55.0025902Z ##[debug]Processed: ##vso[task.issue type=error;]The current Compose file version is not compatible with your engine version. Please upgrade your Compose file to a more recent version, or set a COMPOSE_API_VERSION in your environment.
2023-02-19T08:32:55.0117852Z ##[debug]task result: Failed
2023-02-19T08:32:55.0119880Z ##[error]The process 'C:\ProgramData\Chocolatey\bin\docker-compose.exe' failed with exit code 1

Specifically the line
##[error]The process 'C:\ProgramData\Chocolatey\bin\docker-compose.exe' failed with exit code 1

Printing the environment variables:

C:\agents\2.217.2\externals\git\cmd;
C:\agents\2.217.2\externals\git\mingw64\bin;
C:\Program Files\MongoDB\Server\5.0\bin;
C:\aliyun-cli;
C:\vcpkg;
C:\cf-cli;
C:\Program Files (x86)\NSIS\;
C:\tools\zstd;
C:\Program Files\Mercurial\;
C:\hostedtoolcache\windows\stack\2.9.3\x64;
C:\cabal\bin;
C:\\ghcup\bin;
C:\Program Files\dotnet;
C:\mysql\bin;
C:\Program Files\R\R-4.2.2\bin\x64;
C:\SeleniumWebDrivers\GeckoDriver;
C:\Program Files (x86)\sbt\bin;
C:\Program Files (x86)\GitHub CLI;
C:\Program Files\Git\bin;
C:\Program Files (x86)\pipx_bin;
C:\npm\prefix;
C:\hostedtoolcache\windows\go\1.17.13\x64\bin;
C:\hostedtoolcache\windows\Python\3.7.9\x64\Scripts;
C:\hostedtoolcache\windows\Python\3.7.9\x64;
C:\hostedtoolcache\windows\Ruby\2.5.9\x64\bin;
C:\tools\kotlinc\bin;
C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\8.0.362-9\x64\bin;
C:\Program Files\ImageMagick-7.1.0-Q16-HDRI;
C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin;
C:\ProgramData\kind;
C:\Program Files\Eclipse Foundation\jdk-8.0.302.8-hotspot\bin;
C:\Windows\system32;
C:\Windows;
C:\Windows\System32\Wbem;
C:\Windows\System32\WindowsPowerShell\v1.0\;
C:\Windows\System32\OpenSSH\;
C:\ProgramData\Chocolatey\bin;
C:\Program Files\PowerShell\7\;
C:\Program Files\Microsoft\Web Platform Installer\;
C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;
C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\;
C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;
C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn\;
C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn\;
C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn\;
C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\;
C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn\;
C:\Program Files (x86)\Microsoft SQL Server\160\DTS\Binn\;
C:\Program Files\OpenSSL\bin;
C:\Strawberry\c\bin;
C:\Strawberry\perl\site\bin;
C:\Strawberry\perl\bin;
C:\ProgramData\chocolatey\lib\pulumi\tools\Pulumi\bin;
C:\Program Files\TortoiseSVN\bin;
C:\Program Files\CMake\bin;
C:\ProgramData\chocolatey\lib\maven\apache-maven-3.9.0\bin;
C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;
C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;
C:\Program Files\nodejs\;
C:\Program Files\Git\cmd;
C:\Program Files\Git\mingw64\bin;
C:\Program Files\Git\usr\bin;
C:\Program Files\GitHub CLI\;
c:\tools\php;
C:\Program Files (x86)\sbt\bin;
C:\SeleniumWebDrivers\ChromeDriver\;
C:\SeleniumWebDrivers\EdgeDriver\;
C:\Program Files\Amazon\AWSCLIV2\;
C:\Program Files\Amazon\SessionManagerPlugin\bin\;
C:\Program Files\Amazon\AWSSAMCLI\bin\;
C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin;
C:\Program Files (x86)\Microsoft BizTalk Server\;
C:\Program Files\LLVM\bin;
C:\Users\VssAdministrator\.dotnet\tools;
C:\Users\VssAdministrator\.cargo\bin;
C:\Users\VssAdministrator\AppData\Local\Microsoft\WindowsApps

As can be seen from the environment variable output.
There is no mentioning of the Docker path.

Going ahead to get the version of the Choco Docker Compose via:

- task: PowerShell@2
      inputs:
        targetType: 'inline'
        script: |
          C:\ProgramData\Chocolatey\bin\docker-compose.exe version

Outputted the following:
docker-compose version 1.29.2, build 5becea4c

Choco Docker Compose for reference:
https://community.chocolatey.org/packages/docker-compose

To summarize. It seems that the Docker Compose is installed via Choco package.
And it is currently supports old version. Hence it seems to me, why we have incompatibility issue.
In spite of having latest Docker Compose version listed on the image.

Platforms affected

Azure DevOps
GitHub Actions - Standard Runners
GitHub Actions - Larger Runners

Runner images affected

Image version and build link

Operating System
Microsoft Windows Server 2019
10.0.17763
Runner Image
Image: windows-2019
Version: 20230214.1

Image release:
https://github.com/actions/runner-images/releases/tag/win19%2F20230214.1

Is it regression?

yes. Version 20230129.2

Expected behavior

The Docker container with Docker Compose should run without having incompatibility issue between the Docker compose and Docker engine

Actual behavior

Currently, we have the following error generated when trying to run Docker compose

2023-02-19T08:32:55.0023243Z ##[error]The current Compose file version is not compatible with your engine version. Please upgrade your Compose file to a more recent version, or set a COMPOSE_API_VERSION in your environment.
2023-02-19T08:32:55.0025902Z ##[debug]Processed: ##vso[task.issue type=error;]The current Compose file version is not compatible with your engine version. Please upgrade your Compose file to a more recent version, or set a COMPOSE_API_VERSION in your environment.
2023-02-19T08:32:55.0117852Z ##[debug]task result: Failed
2023-02-19T08:32:55.0119880Z ##[error]The process 'C:\ProgramData\Chocolatey\bin\docker-compose.exe' failed with exit code 1

Repro steps

Below are the steps that reproduce the issue:

- task: DockerCompose@0
      displayName: Spin up container
      inputs:
        containerregistrytype: 'Container Registry'
        dockerRegistryEndpoint: 'Schema-BuildReg'
        dockerComposeFile: 'Sources/Tests/Docker/docker-compose.yml'
        action: 'Run a Docker Compose command'
        dockerComposeCommand: 'up'
        arguments: '-d'

My docker-compose file

version: "3.9"
services:
  sql-server-db:
    container_name: my-mssql
    image: my-mssql-image:latest
    ports:
      - "1433:1433"
    environment:
      SA_PASSWORD: "***************"
      ACCEPT_EULA: "Y"

The text was updated successfully, but these errors were encountered:

mikhailkoliada · 2023-02-21T13:17:15Z

Hello @SillyCode, we will take a look

sergei-pyshnoi · 2023-02-21T14:30:12Z

Hello @SillyCode, we wasn`t able to reproduce this error using docker-compose file with version 3.9. For testing we used more simplified docker compose configuration with windows/nanoserver image (unfortunately I did not find information about image my-mssql-image:latest that you use in your example), and build was successful. I suggest the problem can be with specified container image that you use. If problem still exist, please provide more detailed repro steps (for example minimum repo with ready pipeline and docker-compose file).

docker-compose.yml

version: "3.9"
services:
  sql-server-db:
    container_name: my-mssql
    image: mcr.microsoft.com/windows/nanoserver:1809-KB5022840-amd64
    ports:
      - "1433:1433"
    environment:
      SA_PASSWORD: "***************"
      ACCEPT_EULA: "Y"

Starting: Initialize job logs

Starting: Initialize job
Agent name: 'Hosted Agent'
Agent machine name: 'WIN-D0QP9LQROBF'
Current agent version: '2.217.2'
Operating System
Microsoft Windows Server 2019
10.0.17763
Datacenter
Runner Image
Image: windows-2019
Version: 20230214.1
Included Software: https://github.com/actions/runner-images/blob/win19/20230214.1/images/win/Windows2019-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/win19%2F20230214.1

Spin up container job log

Creating network "azuritetest_default" with the default driver
Pulling sql-server-db (mcr.microsoft.com/windows/nanoserver:1809-KB5022840-amd64)...
1809-KB5022840-amd64: Pulling from windows/nanoserver
Digest: sha256:4fa54f9de5b106a5695b3227a1cceb2a35fee92b0e922b4010691a67d600cbd2
Status: Downloaded newer image for mcr.microsoft.com/windows/nanoserver:1809-KB5022840-amd64
Creating my-mssql ... 

Creating my-mssql ... done

Finishing: Spin up container

SillyCode · 2023-02-21T15:00:11Z

Hi sergei-pyshnoi,
Thank you for having some time for this.

I use a private private registry image that is based on mssql 2019.
It was working fine until the Docker upgrade. Have to mention that no changes to pipeline were made since.

Can you tell which service runs the Docker-compose in the log?
In my case it is located in Chocolatey\bin

##[debug]which 'docker-compose'
##[debug]found: 'C:\ProgramData\Chocolatey\bin\docker-compose.exe'

sergei-pyshnoi · 2023-02-21T15:11:01Z

@SillyCode

C:\ProgramData\Chocolatey\bin\docker-compose.exe -f D:\a\1\s\docker-compose.yaml -f C:\agents\2.217.2\.docker-compose.1676992147189.yml -p AzuriteTest up -d
Creating network "azuritetest_default" with the default driver
Pulling sql-server-db (mcr.microsoft.com/windows/nanoserver:1809-KB5022840-amd64)...
1809-KB5022840-amd64: Pulling from windows/nanoserver
Digest: sha256:4fa54f9de5b106a5695b3227a1cceb2a35fee92b0e922b4010691a67d600cbd2
Status: Downloaded newer image for mcr.microsoft.com/windows/nanoserver:1809-KB5022840-amd64
Creating my-mssql ... 

Creating my-mssql ... done

##[debug]Exit code 0 received from tool 'C:\ProgramData\Chocolatey\bin\docker-compose.exe'

SillyCode · 2023-02-22T12:53:03Z

I am at lost to why does it work.
You can also see mathias-kogler reply

And I can confirm that his workaround works.
I believe there is something wrong with the DockerCompose@0 task. It should take the latest Docker-Compose utility instead of the old one from Choco, that is incompatible

sergei-pyshnoi · 2023-02-22T17:29:25Z

Hello @SillyCode . You are right, the DockerCompose@0 use version from C:\ProgramData\Chocolatey\bin\ . You can create issue in tasks repo about this, because its out of scope of runner-images repo.

SillyCode · 2023-02-23T05:46:52Z

Thank you @sergei-pyshnoi .
I will.

SillyCode added bug report needs triage labels Feb 21, 2023

mikhailkoliada assigned mikhailkoliada and sergei-pyshnoi and unassigned mikhailkoliada Feb 21, 2023

mikhailkoliada added Area: Containers OS: Windows and removed needs triage labels Feb 21, 2023

SillyCode closed this as completed Feb 23, 2023

mikhailkoliada mentioned this issue Mar 1, 2023

Docker compose not working in windows-2019 agent pool. Checked with all the methods still no progress. Can you suggest me how to overcome this issue #7211

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker compose is incompatible with the new Docker engine #7162

Docker compose is incompatible with the new Docker engine #7162

SillyCode commented Feb 21, 2023 •

edited

Loading

mikhailkoliada commented Feb 21, 2023

sergei-pyshnoi commented Feb 21, 2023

SillyCode commented Feb 21, 2023 •

edited

Loading

sergei-pyshnoi commented Feb 21, 2023

SillyCode commented Feb 22, 2023

sergei-pyshnoi commented Feb 22, 2023

SillyCode commented Feb 23, 2023

Docker compose is incompatible with the new Docker engine #7162

Docker compose is incompatible with the new Docker engine #7162

Comments

SillyCode commented Feb 21, 2023 • edited Loading

Description

Platforms affected

Runner images affected

Image version and build link

Is it regression?

Expected behavior

Actual behavior

Repro steps

mikhailkoliada commented Feb 21, 2023

sergei-pyshnoi commented Feb 21, 2023

SillyCode commented Feb 21, 2023 • edited Loading

sergei-pyshnoi commented Feb 21, 2023

SillyCode commented Feb 22, 2023

sergei-pyshnoi commented Feb 22, 2023

SillyCode commented Feb 23, 2023

SillyCode commented Feb 21, 2023 •

edited

Loading

SillyCode commented Feb 21, 2023 •

edited

Loading