Mosaic Augmentation for Object Detection

[innat]

Mosaic Augmentation

• Paper: YOLOv4: Optimal Speed and Accuracy of Object Detection (Figure 3).
• Yolo-V4 Citation: 3718

What is it?
The idea is to take 4 random samples and create a single sample of mosaic fashion with them. For example:

Starter

• PyTorch Implementation:
• TensorFlow Implementation: google/automl/efficientdet/aug/mosaic.py

[bhack]

I think we have this already in the official model garden:
https://github.com/tensorflow/models/blob/master/official/vision/beta/projects/yolo/ops/preprocessing_ops.py#L236
https://github.com/tensorflow/models/blob/master/official/vision/beta/projects/yolo/ops/mosaic.py

[innat]

Oh, I see. Does it mean that we should use these from the tensorflow/models garden or all those relevant operations will be moved here to strengthen the high-level api? Also why the mosaic.py file is under /yolo/ops/ directory? Shouldn't it be more general for all object detection models?

[bhack]

This is the old long story about where to collect reusable CV components.
I suppose that we could port these from the model garden to keras-cv.

[LukeWood]

Think this looks like another good preprocessing layer.

I suppose that we could port these from the model garden to keras-cv.

yeah I agree, it's beginning to look like we are probably going to have a lot of code duplication between keras-cv and model garden. In my opinion this is to be expected - We do not want keras-cv to depend on model garden, but some of the functionality is the same.

@bhack thank you for continuing to point out the existing model garden implementations. They're extremely helpful to me.

[innat]

@bhack @LukeWood @qlzh727
What should be the default target format (BoxMode) for object detection tasks? Will it be pascal format or coco format? In the object detection preprocessing layer, is it expected that the layer should support both formats as input?

[innat]

@LukeWood could you please see the above query? I would most likely send a PR for mosaic if possible.

[LukeWood]

Good question, probably corners? This can have its own issue to discuss. I'll re-open!