Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Refactor] Refactor YOLOX-Pose into mmpose core package #2620

Merged
merged 26 commits into from
Sep 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions configs/body_2d_keypoint/yoloxpose/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss

<!-- [ALGORITHM] -->

<details>
<summary align="right"><a href="https://arxiv.org/abs/2204.06806">YOLO-Pose (CVPRW'2022)</a></summary>

```bibtex
@inproceedings{maji2022yolo,
title={Yolo-pose: Enhancing yolo for multi person pose estimation using object keypoint similarity loss},
author={Maji, Debapriya and Nagori, Soyeb and Mathew, Manu and Poddar, Deepak},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={2637--2646},
year={2022}
}
```

</details>

YOLO-Pose is a bottom-up pose estimation approach that simultaneously detects all person instances and regresses keypoint locations in a single pass.

We implement **YOLOX-Pose** based on the **YOLOX** object detection framework and inherits the benefits of unified pose estimation and object detection from YOLO-pose. To predict keypoint locations more accurately, separate branches with adaptive convolutions are used to regress the offsets for different joints. This allows optimizing the feature extraction for each keypoint.
59 changes: 59 additions & 0 deletions configs/body_2d_keypoint/yoloxpose/coco/yoloxpose_coco.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
<!-- [ALGORITHM] -->

<details>
<summary align="right"><a href="https://arxiv.org/abs/2204.06806">YOLO-Pose (CVPRW'2022)</a></summary>

```bibtex
@inproceedings{maji2022yolo,
title={Yolo-pose: Enhancing yolo for multi person pose estimation using object keypoint similarity loss},
author={Maji, Debapriya and Nagori, Soyeb and Mathew, Manu and Poddar, Deepak},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={2637--2646},
year={2022}
}
```

</details>

<!-- [BACKBONE] -->

<details>
<summary align="right"><a href="https://arxiv.org/abs/2107.08430">YOLOX</a></summary>

```bibtex
@article{ge2021yolox,
title={Yolox: Exceeding yolo series in 2021},
author={Ge, Zheng and Liu, Songtao and Wang, Feng and Li, Zeming and Sun, Jian},
journal={arXiv preprint arXiv:2107.08430},
year={2021}
}
```

</details>

<!-- [DATASET] -->

<details>
<summary align="right"><a href="https://link.springer.com/chapter/10.1007/978-3-319-10602-1_48">COCO (ECCV'2014)</a></summary>

```bibtex
@inproceedings{lin2014microsoft,
title={Microsoft coco: Common objects in context},
author={Lin, Tsung-Yi and Maire, Michael and Belongie, Serge and Hays, James and Perona, Pietro and Ramanan, Deva and Doll{\'a}r, Piotr and Zitnick, C Lawrence},
booktitle={European conference on computer vision},
pages={740--755},
year={2014},
organization={Springer}
}
```

</details>

Results on COCO val2017 with detector having human AP of 56.4 on COCO val2017 dataset

| Arch | Input Size | AP | AP<sup>50</sup> | AP<sup>75</sup> | AR | AR<sup>50</sup> | ckpt | log |
| :-------------------------------------------- | :--------: | :---: | :-------------: | :-------------: | :---: | :-------------: | :-------------------------------------------: | :-------------------------------------------: |
| [yoloxpose_tiny](/configs/body_2d_keypoint/yoloxpose/coco/yoloxpose_tiny_4xb64-300e_coco-416.py) | 416x416 | 0.527 | 0.794 | 0.557 | 0.577 | 0.843 | [ckpt](https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/yolox_pose/yoloxpose_tiny_4xb64-300e_coco-416-76eb44ca_20230829.pth) | [log](https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/yolox_pose/yoloxpose_tiny_4xb64-300e_coco-416-20230829.json) |
| [yoloxpose_s](/configs/body_2d_keypoint/yoloxpose/coco/yoloxpose_s_8xb32-300e_coco-640.py) | 640x640 | 0.642 | 0.873 | 0.702 | 0.688 | 0.912 | [ckpt](https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/yolox_pose/yoloxpose_s_8xb32-300e_coco-640-56c79c1f_20230829.pth) | [log](https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/yolox_pose/yoloxpose_s_8xb32-300e_coco-640-20230829.json) |
| [yoloxpose_m](/configs/body_2d_keypoint/yoloxpose/coco/yoloxpose_m_8xb32-300e_coco-640.py) | 640x640 | 0.697 | 0.903 | 0.766 | 0.739 | 0.933 | [ckpt](https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/yolox_pose/yoloxpose_m_8xb32-300e_coco-640-84e9a538_20230829.pth) | [log](https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/yolox_pose/yoloxpose_m_8xb32-300e_coco-640-20230829.json) |
| [yoloxpose_l](/configs/body_2d_keypoint/yoloxpose/coco/yoloxpose_l_8xb32-300e_coco-640.py) | 640x640 | 0.714 | 0.906 | 0.785 | 0.756 | 0.934 | [ckpt](https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/yolox_pose/yoloxpose_l_8xb32-300e_coco-640-de0f8dee_20230829.pth) | [log](https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/yolox_pose/yoloxpose_l_8xb32-300e_coco-640-20230829.json) |
72 changes: 72 additions & 0 deletions configs/body_2d_keypoint/yoloxpose/coco/yoloxpose_coco.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
Collections:
- Name: YOLOXPose
Paper:
Title: 'YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss'
URL: https://arxiv.org/abs/2204.06806
README: https://github.com/open-mmlab/mmpose/blob/main/docs/src/papers/algorithms/yolopose.md
Models:
- Config: configs/body_2d_keypoint/yoloxpose/coco/yoloxpose_tiny_4xb64-300e_coco-416.py
In Collection: YOLOXPose
Metadata:
Architecture: &id001
- YOLOXPose
Training Data: COCO
Name: yoloxpose_tiny_4xb64-300e_coco-416
Results:
- Dataset: COCO
Metrics:
AP: 0.527
AP@0.5: 0.794
AP@0.75: 0.557
AR: 0.577
AR@0.5: 0.843
Task: Body 2D Keypoint
Weights: https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/yolox_pose/yoloxpose_tiny_4xb64-300e_coco-416-76eb44ca_20230829.pth
- Config: configs/body_2d_keypoint/yoloxpose/coco/yoloxpose_s_8xb32-300e_coco-640.py
In Collection: YOLOXPose
Metadata:
Architecture: *id001
Training Data: COCO
Name: yoloxpose_s_8xb32-300e_coco-640
Results:
- Dataset: COCO
Metrics:
AP: 0.642
AP@0.5: 0.873
AP@0.75: 0.702
AR: 0.688
AR@0.5: 0.912
Task: Body 2D Keypoint
Weights: https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/yolox_pose/yoloxpose_s_8xb32-300e_coco-640-56c79c1f_20230829.pth
- Config: configs/body_2d_keypoint/yoloxpose/coco/yoloxpose_m_8xb32-300e_coco-640.py
In Collection: YOLOXPose
Metadata:
Architecture: *id001
Training Data: COCO
Name: yoloxpose_m_8xb32-300e_coco-640
Results:
- Dataset: COCO
Metrics:
AP: 0.697
AP@0.5: 0.903
AP@0.75: 0.766
AR: 0.739
AR@0.5: 0.933
Task: Body 2D Keypoint
Weights: https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/yolox_pose/yoloxpose_m_8xb32-300e_coco-640-84e9a538_20230829.pth
- Config: configs/body_2d_keypoint/yoloxpose/coco/yoloxpose_l_8xb32-300e_coco-640.py
In Collection: YOLOXPose
Metadata:
Architecture: *id001
Training Data: COCO
Name: yoloxpose_l_8xb32-300e_coco-640
Results:
- Dataset: COCO
Metrics:
AP: 0.714
AP@0.5: 0.906
AP@0.75: 0.785
AR: 0.756
AR@0.5: 0.934
Task: Body 2D Keypoint
Weights: https://download.openmmlab.com/mmpose/v1/body_2d_keypoint/yolox_pose/yoloxpose_l_8xb32-300e_coco-640-de0f8dee_20230829.pth
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
_base_ = './yolopose_s_8xb32-300e_coco-640.py'

widen_factor = 1
deepen_factor = 1
checkpoint = 'https://download.openmmlab.com/mmdetection/v2.0/yolox/yolox_' \
'l_8x8_300e_coco/yolox_l_8x8_300e_coco_20211126_140236-d3bd2b23.pth'

# model settings
model = dict(
backbone=dict(
deepen_factor=deepen_factor,
widen_factor=widen_factor,
init_cfg=dict(checkpoint=checkpoint),
),
neck=dict(
in_channels=[256, 512, 1024], out_channels=256, num_csp_blocks=3),
head=dict(head_module_cfg=dict(widen_factor=widen_factor)))
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
_base_ = './yolopose_s_8xb32-300e_coco-640.py'

widen_factor = 0.75
deepen_factor = 0.67
checkpoint = 'https://download.openmmlab.com/mmpose/v1/pretrained_models/' \
'yolox_m_8x8_300e_coco_20230829.pth'

# model settings
model = dict(
backbone=dict(
deepen_factor=deepen_factor,
widen_factor=widen_factor,
init_cfg=dict(checkpoint=checkpoint),
),
neck=dict(in_channels=[192, 384, 768], out_channels=192, num_csp_blocks=2),
head=dict(head_module_cfg=dict(widen_factor=widen_factor)))
Loading
Loading