Hole cutting in polygons

[nstolyarov]

My actions before raising this issue

• Read/searched the docs
• Searched past issues

There is no way to cut holes in polygons. Sometimes there may be a foreign object on the surface to be marked, which must be removed. Now this can only be done using the complex action shown in the video. The same video shows an example of how this is done in other software. An attempt to add a separate class for marking in black (RGB 000) also failed, because CVAT does not display black on the result mask.

Expected Behaviour

Get the ability to cut holes in polygons.

Current Behaviour

No normal ability to cut holes in polygons.

Possible Solution

This can be done, for example, by adding an operation to subtract one polygon from another. Alternatively, you can add the Background class, which can be used to mark up objects that are not subject to markup.

Context

Cutting is a very common operation in our project. The current cutting method is very time consuming.

Your Environment

• Git hash commit (git log -1): commit 7a2dcff (HEAD -> develop, origin/develop, origin/HEAD)
  Author: Anastasia Yasakova anastasia.yasakova@intel.com
  Date: Tue Mar 9 10:41:33 2021 +0300
• Docker version docker version (e.g. Docker 17.0.05): 20.10.5
• Are you using Docker Swarm or Kubernetes? No
• Operating System and version (e.g. Linux, Windows, MacOS): GNU/Linux Ubuntu
  4.15.0-136-generic
  x86_64
• Video: https://drive.google.com/file/d/1MxlHa8XMlC__-w4JoKXcQG0uDzaztdY6/view?usp=sharing

[bsekachev]

Hi, @nstolyarov

Please, use additional "background" class to create "holes" in polygons. Just draw it over a polygon object and then dump .png masks.

The idea is that polygon with a hole is not a polygon anymore, it is a mask or another complex object (we cannot represent holes using only coordinate number sequences). CVAT does not support masks and I do not expect it will in the nearest future (if only community is ready to contribute).

Currently I will close the issue because it is dublicate of #1849 and #2384.

[nstolyarov]

@bsekachev, thank you for answer. But as I said there is the problem with "background" class. It seems reasonable to use it, but when I add a label "background" (or any other name) with RGB(0,0,0) any other classes override this one on SegmentationClass mask if we dump annotations via Segmentation mask 1.1. Even if I move it to foreground with "+" hotkey.

[bsekachev]

@zhiltsov-max Could you please comment on this?

I also tried to dump a task with a couple of polygons and in case of dumping to CVAT Images it works as expected, but in case of dumping to SegmentationMask I get an archive without any masks.

task_test-2021_03_17_07_58_07-segmentation mask 1.1.zip

[nstolyarov]

Here is example:

First three images show layers

Actual segmentation. "background" label has RGB(0,0,0) color.

Result of Segmentation mask 1.1 dump

SegmentationClass

SegmentationObject

[nstolyarov]

Here is the second example but I have label "Trash" with RGB(1,1,1) instead of "background" RGB(0,0,0)

SegmentationClass

SegmentationObject

It can work but you need to use other tools for changing pixels color after dump. Maybe it is not a bad idea to make "background" label be able in foreground layer?

[zhiltsov-max]

Currently in the develop branch it should be working this way:

Any class with the name background or with color (0, 0, 0) should fit. The "background" polygon was moved to foreground (+ or = key). The order of annotations is preserved, so being the latter annotation will work the same way.

SegmentationClass

SegmentationObject

Importing produces duplicated polygons.

[nstolyarov]

@zhiltsov-max could you please provide your commit id?

[zhiltsov-max]

83a08cb

[zhiltsov-max]

@nstolyarov, can you check if the fix in openvinotoolkit/datumaro#188 resolves the problem for you? It should allow exporting of masks with background class polygons on top of the other shapes to create holes in them. It should now work with both SegmentationObject and SegmentationInstance directories, instead of only SegmentationObject.

How to test:

1. Change this line: https://github.com/openvinotoolkit/cvat/blob/develop/cvat/requirements/base.txt#L53
   to git+https://github.com/openvinotoolkit/datumaro@zm/holes-in-polygons
2. Rebuild the container
3. Restart the server

Examples:

I have a task with such config:

(where background has color #000)

Annotation window

Exported masks
SegmentationClass

SegmentationObject

(But they are different instances yet)

But you're right about high overhead of using this functionality.

[nstolyarov]

@zhiltsov-max, hi! I tried to do it but unfortunately I still have no "background" over objects of other classes.

My steps:

1. I changed this line: https://github.com/openvinotoolkit/cvat/blob/develop/cvat/requirements/base.txt#L53 to git+https://github.com/openvinotoolkit/datumaro@zm/holes-in-polygons
2. I removed docker containers and images with docker rm $(docker ps -a -q) and docker rmi $(docker images -q) respectively
3. I rebuild and started the container with docker-compose up --force-recreate

Maybe I do something wrong when fully rebuilding containers?

[zhiltsov-max]

@nstolyarov, in step 3 try to rebuild with docker-compose -f ./docker-compose.yml -f ./docker-compose.dev.yml up --build

[nstolyarov]

@zhiltsov-max hi!
Thank you for this fix. Seems it works in the #3030 version. But is it ok that on instance mask "background" still has object labeling?

[zhiltsov-max]

@nstolyarov, thanks for testing and confirming it works in the develop branch of CVAT. The patch I mentioned above (openvinotoolkit/datumaro#188) fixes the problem with instance masks. This patch is applied to the Datumaro subproject, which is used by CVAT, so it doesn't affect CVAT directly and requires an extra step (№1 above) to test it, unless it is applied and merged into CVAT.