Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to Docker Compose V2 #5524

Merged
merged 3 commits into from
Dec 29, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -255,14 +255,14 @@ jobs:
HOST_COVERAGE_DATA_DIR: ${{ github.workspace }}
CONTAINER_COVERAGE_DATA_DIR: "/coverage_data"
run: |
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d cvat_opa cvat_server
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d cvat_opa cvat_server
max_tries=12
while [[ $(curl -s -o /dev/null -w "%{http_code}" localhost:8181/health?bundles) != "200" && max_tries -gt 0 ]]; do (( max_tries-- )); sleep 5; done

docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
docker compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
-c 'python manage.py test cvat/apps -v 2'

docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
docker compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
-c 'yarn --frozen-lockfile --ignore-scripts && yarn workspace cvat-core run test'

- name: Creating a log file from cvat containers
Expand Down Expand Up @@ -325,7 +325,7 @@ jobs:

- name: Run CVAT instance
run: |
docker-compose \
docker compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
-f components/serverless/docker-compose.serverless.yml \
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -222,15 +222,15 @@ jobs:
HOST_COVERAGE_DATA_DIR: ${{ github.workspace }}
CONTAINER_COVERAGE_DATA_DIR: "/coverage_data"
run: |
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d cvat_opa cvat_server
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d cvat_opa cvat_server

max_tries=12
while [[ $(curl -s -o /dev/null -w "%{http_code}" localhost:8181/health?bundles) != "200" && max_tries -gt 0 ]]; do (( max_tries-- )); sleep 5; done

docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
docker compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
-c 'python manage.py test cvat/apps -v 2'

docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
docker compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
-c 'yarn --frozen-lockfile --ignore-scripts && yarn workspace cvat-core run test'

- name: Creating a log file from cvat containers
Expand Down Expand Up @@ -292,7 +292,7 @@ jobs:

- name: Run CVAT instance
run: |
docker-compose \
docker compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
-f components/serverless/docker-compose.serverless.yml \
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish_docker_images.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
- uses: actions/checkout@v3
- name: Build images
run: |
CVAT_VERSION=latest CLAM_AV=yes INSTALL_SOURCES=yes docker-compose -f docker-compose.yml -f docker-compose.dev.yml build
CVAT_VERSION=latest CLAM_AV=yes INSTALL_SOURCES=yes docker compose -f docker-compose.yml -f docker-compose.dev.yml build
- name: Login to Docker Hub
uses: docker/login-action@v1
with:
Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/schedule.yml
Original file line number Diff line number Diff line change
Expand Up @@ -246,17 +246,17 @@ jobs:
HOST_COVERAGE_DATA_DIR: ${{ github.workspace }}
CONTAINER_COVERAGE_DATA_DIR: "/coverage_data"
run: |
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d cvat_opa cvat_server
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d cvat_opa cvat_server
max_tries=12
while [[ $(curl -s -o /dev/null -w "%{http_code}" localhost:8181/health?bundles) != "200" && max_tries -gt 0 ]]; do (( max_tries-- )); sleep 5; done

docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
docker compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
-c 'coverage run -a manage.py test cvat/apps && mv .coverage ${CONTAINER_COVERAGE_DATA_DIR}'

docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
docker compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml run cvat_ci /bin/bash \
-c 'yarn --frozen-lockfile --ignore-scripts && yarn workspace cvat-core run test'

docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml down -v
docker compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.ci.yml down -v

- name: Uploading code coverage results as an artifact
uses: actions/upload-artifact@v3.1.1
Expand Down Expand Up @@ -318,7 +318,7 @@ jobs:

- name: Run CVAT instance
run: |
docker-compose \
docker compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
-f tests/docker-compose.file_share.yml \
Expand Down Expand Up @@ -458,13 +458,13 @@ jobs:
run: |
npx nyc report --reporter=text-lcov >> ${HOST_COVERAGE_DATA_DIR}/lcov.info

docker-compose \
docker compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
-f docker-compose.ci.yml \
run cvat_ci /bin/bash -c 'cd ${CONTAINER_COVERAGE_DATA_DIR} && coveralls-lcov -v -n lcov.info > ${CONTAINER_COVERAGE_DATA_DIR}/coverage.json'

docker-compose \
docker compose \
-f docker-compose.yml \
-f docker-compose.dev.yml \
-f docker-compose.ci.yml \
Expand Down
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Filename pattern to simplify uploading cloud storage data for a task (<https://github.com/opencv/cvat/pull/5498>)

### Changed
- TDB
- The Docker Compose files now use the Compose Specification version
of the format. This version is supported by Docker Compose 1.27.0+
(<https://github.com/opencv/cvat/pull/5524>).

### Deprecated
- TDB
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ Here are some screencasts showing how to use CVAT.
[Computer Vision Annotation Course](https://www.youtube.com/playlist?list=PL0to7Ng4PuuYQT4eXlHb_oIlq_RPeuasN): we introduce our course series designed to help you annotate data faster and better using CVAT. This course is about CVAT deployment and integrations, it includes presentations and covers the following topics:

- **Speeding up your data annotation process: introduction to CVAT and Datumaro**. What problems do CVAT and Datumaro solve, and how they can speed up your model training process. Some resources you can use to learn more about how to use them.
- **Deployment and use CVAT**. Use the app online at [app.cvat.ai](app.cvat.ai). A local deployment. A containerized local deployment with docker-compose (for regular use), and a local cluster deployment with Kubernetes (for enterprise users). A 2-minute tour of the interface, a breakdown of CVAT’s internals, and a demonstration of how to deploy CVAT using docker-compose.
- **Deployment and use CVAT**. Use the app online at [app.cvat.ai](app.cvat.ai). A local deployment. A containerized local deployment with Docker Compose (for regular use), and a local cluster deployment with Kubernetes (for enterprise users). A 2-minute tour of the interface, a breakdown of CVAT’s internals, and a demonstration of how to deploy CVAT using Docker Compose.

[Product tour](https://www.youtube.com/playlist?list=PL0to7Ng4Puua37NJVMIShl_pzqJTigFzg): in this course, we show how to use CVAT, and help to get familiar with CVAT functionality and interfaces. This course does not cover integrations and is dedicated solely to CVAT. It covers the following topics:

Expand Down
1 change: 0 additions & 1 deletion components/analytics/docker-compose.analytics.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
version: '3.3'
services:
elasticsearch:
container_name: cvat_elasticsearch
Expand Down
2 changes: 1 addition & 1 deletion components/serverless/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@

```bash
# From project root directory
docker-compose -f docker-compose.yml -f components/serverless/docker-compose.serverless.yml up -d
docker compose -f docker-compose.yml -f components/serverless/docker-compose.serverless.yml up -d
```
1 change: 0 additions & 1 deletion components/serverless/docker-compose.serverless.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
version: '3.3'
services:
nuclio:
container_name: nuclio
Expand Down
1 change: 0 additions & 1 deletion docker-compose.ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
#
# SPDX-License-Identifier: MIT
#
version: '3.3'

services:
cvat_ci:
Expand Down
1 change: 0 additions & 1 deletion docker-compose.dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
#
# SPDX-License-Identifier: MIT
#
version: '3.3'

services:
cvat_server:
Expand Down
2 changes: 0 additions & 2 deletions docker-compose.https.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@
#
# SPDX-License-Identifier: MIT

version: '3.3'

services:
cvat_server:
labels:
Expand Down
2 changes: 0 additions & 2 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@
#
# SPDX-License-Identifier: MIT

version: '3.3'

services:
cvat_db:
container_name: cvat_db
Expand Down
2 changes: 1 addition & 1 deletion helm-chart/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ cvat:
# - mountPath: /tmp
# name: tmp
# subPath: test
composeCompatibleServiceName: true # Sets service name to opa in order to be docker-compose compatible. Necessary because changing IAM_OPA_DATA_URL via environment variables in current images. Hinders multiple deployment due to duplicate name
composeCompatibleServiceName: true # Sets service name to opa in order to be compatible with Docker Compose. Necessary because changing IAM_OPA_DATA_URL via environment variables in current images. Hinders multiple deployment due to duplicate name
service:
type: ClusterIP
ports:
Expand Down
4 changes: 2 additions & 2 deletions site/content/en/docs/administration/advanced/analytics.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,14 @@ It is possible to proxy annotation logs from client to ELK. To do that run the f

```bash
# From project root directory
docker-compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml build
docker compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml build
```

### Run docker container

```bash
# From project root directory
docker-compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml up -d
docker compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml up -d
```

At the moment it is not possible to save advanced settings. Below values should be specified manually.
Expand Down
8 changes: 4 additions & 4 deletions site/content/en/docs/administration/advanced/backup_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,10 @@ Docker volumes are used to store all CVAT data:
All CVAT containers should be stopped before backup:

```shell
docker-compose stop
docker compose stop
```

Please don't forget to include all the compose config files that were used in the docker-compose command
Please don't forget to include all the compose config files that were used in the `docker compose` command
using the `-f` parameter.

Backup data:
Expand Down Expand Up @@ -66,7 +66,7 @@ Note: CVAT containers must exist (if no, please follow the [installation guide](
Stop all CVAT containers:

```shell
docker-compose stop
docker compose stop
```

Restore data:
Expand All @@ -82,7 +82,7 @@ docker run --rm --name temp_backup --volumes-from cvat_elasticsearch -v $(pwd):/
After that run CVAT as usual:

```shell
docker-compose up -d
docker compose up -d
```

## Additional resources
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,21 +11,21 @@ description: 'Information about the installation of components needed for semi-a

<!--lint disable maximum-line-length-->

> **⚠ WARNING: Do not use `docker-compose up`**
> If you did, make sure all containers are stopped by `docker-compose down`.
> **⚠ WARNING: Do not use `docker compose up`**
> If you did, make sure all containers are stopped by `docker compose down`.

- To bring up cvat with auto annotation tool, from cvat root directory, you need to run:

```bash
docker-compose -f docker-compose.yml -f components/serverless/docker-compose.serverless.yml up -d
docker compose -f docker-compose.yml -f components/serverless/docker-compose.serverless.yml up -d
```

If you did any changes to the docker-compose files, make sure to add `--build` at the end.
If you did any changes to the Docker Compose files, make sure to add `--build` at the end.

To stop the containers, simply run:

```bash
docker-compose -f docker-compose.yml -f components/serverless/docker-compose.serverless.yml down
docker compose -f docker-compose.yml -f components/serverless/docker-compose.serverless.yml down
```

- You have to install `nuctl` command line tool to build and deploy serverless
Expand All @@ -47,7 +47,7 @@ description: 'Information about the installation of components needed for semi-a

- Create `cvat` project inside nuclio dashboard where you will deploy new serverless functions
and deploy a couple of DL models. Commands below should be run only after CVAT has been installed
using `docker-compose` because it runs nuclio dashboard which manages all serverless functions.
using `docker compose` because it runs nuclio dashboard which manages all serverless functions.

```bash
nuctl create project cvat
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ Before starting, ensure that the following prerequisites are met:

- Let's build custom elasticsearch, logstash and kibana images with the following command
```shell
docker-compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml build
docker compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml build
```

- Tag images:
Expand Down
18 changes: 9 additions & 9 deletions site/content/en/docs/administration/advanced/upgrade_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,16 @@ To upgrade CVAT, follow these steps:

- Go to the previously cloned CVAT directory and stop all CVAT containers with:
```shell
docker-compose down
docker compose down
```
If you have included [additional components](/docs/administration/basics/installation/#additional-components),
include all compose configuration files that are used, e.g.:
```shell
docker-compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml down
docker compose -f docker-compose.yml -f components/analytics/docker-compose.analytics.yml down
```

- Update CVAT source code by any preferable way: clone with git or download zip file from GitHub.
Note that you need to download the entire source code, not just the `docker-compose` configuration file.
Note that you need to download the entire source code, not just the Docker Compose configuration file.
Check the
[installation guide](/docs/administration/basics/installation/#how-to-get-cvat-source-code) for details.

Expand All @@ -44,7 +44,7 @@ To upgrade CVAT, follow these steps:

- Start CVAT with:
```shell
docker-compose up -d
docker compose up -d
```
When CVAT starts, it will upgrade its DB in accordance with the latest schema.
It can take time especially if you have a lot of data.
Expand All @@ -60,7 +60,7 @@ Step by step commands how to udgrade CVAT from v1.7.0 to v2.1.0.
Let's assume that you have CVAT v1.7.0 working.
```shell
cd cvat
docker-compose down
docker compose down
cd ..
mv cvat cvat_old
wget https://github.com/opencv/cvat/archive/refs/tags/v2.1.0.zip
Expand All @@ -70,7 +70,7 @@ docker pull cvat/server:v2.1.0
docker tag cvat/server:v2.1.0 openvino/cvat_server:latest
docker pull cvat/ui:v2.1.0
docker tag cvat/ui:v2.1.0 openvino/cvat_ui:latest
docker-compose up -d
docker compose up -d
```
## How to upgrade PostgreSQL database base image

Expand All @@ -86,7 +86,7 @@ docker-compose up -d

1. Stop CVAT:
```shell
docker-compose down
docker compose down
```

1. Delete current PostrgeSQL’s volume, that's why it's important to have a backup:
Expand All @@ -100,7 +100,7 @@ docker-compose up -d

1. Start database container only:
```shell
docker-compose up -d cvat_db
docker compose up -d cvat_db
```

1. Import PostgreSQL dump into new DB container:
Expand All @@ -110,5 +110,5 @@ docker-compose up -d

1. Start CVAT:
```shell
docker-compose up -d
docker compose up -d
```
Loading