[Feature] Add shelf and campus datasets (open-mmlab#1225)

* [Feature] Add shelf and campus datasets * fix config typo * Add unittests for dataset and pipeline and modify configs * unittest for dataset * add more unittests for pose3d_transform * move several general functions to base class * add related docs for campus and shelf dataset * modify config for campus * fix typo * fix some docs * remove _get_db method in base class * update log path * fix log path * add more descriptions in configs * keep _get_db method in base class and add error message in subclasses * add explanation of some functions * add explanation about dataset in doc string * modify the path for train_pose_db_file in config and docs * add code license of the dataset
ajgrafton · Apr 2, 2022 · 81092da · 81092da
1 parent 36eb886
commit 81092da
Show file tree

Hide file tree

Showing 33 changed files with 3,774 additions and 48 deletions.
diff --git a/README.md b/README.md
@@ -161,7 +161,7 @@ A summary can be found in the [Model Zoo](https://mmpose.readthedocs.io/en/lates
 * [x] [MPII](https://mmpose.readthedocs.io/en/latest/papers/datasets.html#mpii-cvpr-2014) \[[homepage](http://human-pose.mpi-inf.mpg.de/)\] (CVPR'2014)
 * [x] [Human3.6M](https://mmpose.readthedocs.io/en/latest/papers/datasets.html#human3-6m-tpami-2014) \[[homepage](http://vision.imar.ro/human3.6m/description.php)\] (TPAMI'2014)
 * [x] [COCO](https://mmpose.readthedocs.io/en/latest/papers/datasets.html#coco-eccv-2014) \[[homepage](http://cocodataset.org/)\] (ECCV'2014)
-* [x] [CMU Panoptic](https://mmpose.readthedocs.io/en/latest/papers/datasets.html#cmu-panoptic-iccv-2015) (ICCV'2015)
+* [x] [CMU Panoptic](https://mmpose.readthedocs.io/en/latest/papers/datasets.html#cmu-panoptic-iccv-2015) \[[homepage](http://domedb.perception.cs.cmu.edu/)\] (ICCV'2015)
 * [x] [DeepFashion](https://mmpose.readthedocs.io/en/latest/papers/datasets.html#deepfashion-cvpr-2016) \[[homepage](http://mmlab.ie.cuhk.edu.hk/projects/DeepFashion/LandmarkDetection.html)\] (CVPR'2016)
 * [x] [300W](https://mmpose.readthedocs.io/en/latest/papers/datasets.html#300w-imavis-2016) \[[homepage](https://ibug.doc.ic.ac.uk/resources/300-W/)\] (IMAVIS'2016)
 * [x] [RHD](https://mmpose.readthedocs.io/en/latest/papers/datasets.html#rhd-iccv-2017) \[[homepage](https://lmb.informatik.uni-freiburg.de/resources/datasets/RenderedHandposeDataset.en.html)\] (ICCV'2017)

diff --git a/configs/_base_/datasets/campus.py b/configs/_base_/datasets/campus.py
@@ -0,0 +1,151 @@
+dataset_info = dict(
+    dataset_name='campus',
+    paper_info=dict(
+        author='Belagiannis, Vasileios and Amin, Sikandar and Andriluka, '
+        'Mykhaylo and Schiele, Bernt and Navab, Nassir and Ilic, Slobodan',
+        title='3D Pictorial Structures for Multiple Human Pose Estimation',
+        container='IEEE Computer Society Conference on Computer Vision and '
+        'Pattern Recognition (CVPR)',
+        year='2014',
+        homepage='http://campar.in.tum.de/Chair/MultiHumanPose',
+    ),
+    keypoint_info={
+        0:
+        dict(
+            name='right_ankle',
+            id=0,
+            color=[255, 128, 0],
+            type='lower',
+            swap='left_ankle'),
+        1:
+        dict(
+            name='right_knee',
+            id=1,
+            color=[255, 128, 0],
+            type='lower',
+            swap='left_knee'),
+        2:
+        dict(
+            name='right_hip',
+            id=2,
+            color=[255, 128, 0],
+            type='lower',
+            swap='left_hip'),
+        3:
+        dict(
+            name='left_hip',
+            id=3,
+            color=[0, 255, 0],
+            type='lower',
+            swap='right_hip'),
+        4:
+        dict(
+            name='left_knee',
+            id=4,
+            color=[0, 255, 0],
+            type='lower',
+            swap='right_knee'),
+        5:
+        dict(
+            name='left_ankle',
+            id=5,
+            color=[0, 255, 0],
+            type='lower',
+            swap='right_ankle'),
+        6:
+        dict(
+            name='right_wrist',
+            id=6,
+            color=[255, 128, 0],
+            type='upper',
+            swap='left_wrist'),
+        7:
+        dict(
+            name='right_elbow',
+            id=7,
+            color=[255, 128, 0],
+            type='upper',
+            swap='left_elbow'),
+        8:
+        dict(
+            name='right_shoulder',
+            id=8,
+            color=[255, 128, 0],
+            type='upper',
+            swap='left_shoulder'),
+        9:
+        dict(
+            name='left_shoulder',
+            id=9,
+            color=[0, 255, 0],
+            type='upper',
+            swap='right_shoulder'),
+        10:
+        dict(
+            name='left_elbow',
+            id=10,
+            color=[0, 255, 0],
+            type='upper',
+            swap='right_elbow'),
+        11:
+        dict(
+            name='left_wrist',
+            id=11,
+            color=[0, 255, 0],
+            type='upper',
+            swap='right_wrist'),
+        12:
+        dict(
+            name='bottom_head',
+            id=12,
+            color=[51, 153, 255],
+            type='upper',
+            swap=''),
+        13:
+        dict(
+            name='top_head',
+            id=13,
+            color=[51, 153, 255],
+            type='upper',
+            swap=''),
+    },
+    skeleton_info={
+        0:
+        dict(link=('right_ankle', 'right_knee'), id=0, color=[255, 128, 0]),
+        1:
+        dict(link=('right_knee', 'right_hip'), id=1, color=[255, 128, 0]),
+        2:
+        dict(link=('left_hip', 'left_knee'), id=2, color=[0, 255, 0]),
+        3:
+        dict(link=('left_knee', 'left_ankle'), id=3, color=[0, 255, 0]),
+        4:
+        dict(link=('right_hip', 'left_hip'), id=4, color=[51, 153, 255]),
+        5:
+        dict(link=('right_wrist', 'right_elbow'), id=5, color=[255, 128, 0]),
+        6:
+        dict(
+            link=('right_elbow', 'right_shoulder'), id=6, color=[255, 128, 0]),
+        7:
+        dict(link=('left_shoulder', 'left_elbow'), id=7, color=[0, 255, 0]),
+        8:
+        dict(link=('left_elbow', 'left_wrist'), id=8, color=[0, 255, 0]),
+        9:
+        dict(link=('right_hip', 'right_shoulder'), id=9, color=[255, 128, 0]),
+        10:
+        dict(link=('left_hip', 'left_shoulder'), id=10, color=[0, 255, 0]),
+        11:
+        dict(
+            link=('right_shoulder', 'bottom_head'), id=11, color=[255, 128,
+                                                                  0]),
+        12:
+        dict(link=('left_shoulder', 'bottom_head'), id=12, color=[0, 255, 0]),
+        13:
+        dict(link=('bottom_head', 'top_head'), id=13, color=[51, 153, 255]),
+    },
+    joint_weights=[
+        1.5, 1.2, 1.0, 1.0, 1.2, 1.5, 1.5, 1.2, 1.0, 1.0, 1.2, 1.5, 1.0, 1.0
+    ],
+    sigmas=[
+        0.089, 0.087, 0.107, 0.107, 0.087, 0.089, 0.062, 0.072, 0.079, 0.079,
+        0.072, 0.062, 0.026, 0.026
+    ])
diff --git a/configs/_base_/datasets/shelf.py b/configs/_base_/datasets/shelf.py
@@ -0,0 +1,151 @@
+dataset_info = dict(
+    dataset_name='shelf',
+    paper_info=dict(
+        author='Belagiannis, Vasileios and Amin, Sikandar and Andriluka, '
+        'Mykhaylo and Schiele, Bernt and Navab, Nassir and Ilic, Slobodan',
+        title='3D Pictorial Structures for Multiple Human Pose Estimation',
+        container='IEEE Computer Society Conference on Computer Vision and '
+        'Pattern Recognition (CVPR)',
+        year='2014',
+        homepage='http://campar.in.tum.de/Chair/MultiHumanPose',
+    ),
+    keypoint_info={
+        0:
+        dict(
+            name='right_ankle',
+            id=0,
+            color=[255, 128, 0],
+            type='lower',
+            swap='left_ankle'),
+        1:
+        dict(
+            name='right_knee',
+            id=1,
+            color=[255, 128, 0],
+            type='lower',
+            swap='left_knee'),
+        2:
+        dict(
+            name='right_hip',
+            id=2,
+            color=[255, 128, 0],
+            type='lower',
+            swap='left_hip'),
+        3:
+        dict(
+            name='left_hip',
+            id=3,
+            color=[0, 255, 0],
+            type='lower',
+            swap='right_hip'),
+        4:
+        dict(
+            name='left_knee',
+            id=4,
+            color=[0, 255, 0],
+            type='lower',
+            swap='right_knee'),
+        5:
+        dict(
+            name='left_ankle',
+            id=5,
+            color=[0, 255, 0],
+            type='lower',
+            swap='right_ankle'),
+        6:
+        dict(
+            name='right_wrist',
+            id=6,
+            color=[255, 128, 0],
+            type='upper',
+            swap='left_wrist'),
+        7:
+        dict(
+            name='right_elbow',
+            id=7,
+            color=[255, 128, 0],
+            type='upper',
+            swap='left_elbow'),
+        8:
+        dict(
+            name='right_shoulder',
+            id=8,
+            color=[255, 128, 0],
+            type='upper',
+            swap='left_shoulder'),
+        9:
+        dict(
+            name='left_shoulder',
+            id=9,
+            color=[0, 255, 0],
+            type='upper',
+            swap='right_shoulder'),
+        10:
+        dict(
+            name='left_elbow',
+            id=10,
+            color=[0, 255, 0],
+            type='upper',
+            swap='right_elbow'),
+        11:
+        dict(
+            name='left_wrist',
+            id=11,
+            color=[0, 255, 0],
+            type='upper',
+            swap='right_wrist'),
+        12:
+        dict(
+            name='bottom_head',
+            id=12,
+            color=[51, 153, 255],
+            type='upper',
+            swap=''),
+        13:
+        dict(
+            name='top_head',
+            id=13,
+            color=[51, 153, 255],
+            type='upper',
+            swap=''),
+    },
+    skeleton_info={
+        0:
+        dict(link=('right_ankle', 'right_knee'), id=0, color=[255, 128, 0]),
+        1:
+        dict(link=('right_knee', 'right_hip'), id=1, color=[255, 128, 0]),
+        2:
+        dict(link=('left_hip', 'left_knee'), id=2, color=[0, 255, 0]),
+        3:
+        dict(link=('left_knee', 'left_ankle'), id=3, color=[0, 255, 0]),
+        4:
+        dict(link=('right_hip', 'left_hip'), id=4, color=[51, 153, 255]),
+        5:
+        dict(link=('right_wrist', 'right_elbow'), id=5, color=[255, 128, 0]),
+        6:
+        dict(
+            link=('right_elbow', 'right_shoulder'), id=6, color=[255, 128, 0]),
+        7:
+        dict(link=('left_shoulder', 'left_elbow'), id=7, color=[0, 255, 0]),
+        8:
+        dict(link=('left_elbow', 'left_wrist'), id=8, color=[0, 255, 0]),
+        9:
+        dict(link=('right_hip', 'right_shoulder'), id=9, color=[255, 128, 0]),
+        10:
+        dict(link=('left_hip', 'left_shoulder'), id=10, color=[0, 255, 0]),
+        11:
+        dict(
+            link=('right_shoulder', 'bottom_head'), id=11, color=[255, 128,
+                                                                  0]),
+        12:
+        dict(link=('left_shoulder', 'bottom_head'), id=12, color=[0, 255, 0]),
+        13:
+        dict(link=('bottom_head', 'top_head'), id=13, color=[51, 153, 255]),
+    },
+    joint_weights=[
+        1.5, 1.2, 1.0, 1.0, 1.2, 1.5, 1.5, 1.2, 1.0, 1.0, 1.2, 1.5, 1.0, 1.0
+    ],
+    sigmas=[
+        0.089, 0.087, 0.107, 0.107, 0.087, 0.089, 0.062, 0.072, 0.079, 0.079,
+        0.072, 0.062, 0.026, 0.026
+    ])
diff --git a/configs/body/3d_kpt_mview_rgb_img/voxelpose/campus/voxelpose_campus.md b/configs/body/3d_kpt_mview_rgb_img/voxelpose/campus/voxelpose_campus.md
@@ -0,0 +1,41 @@
+<!-- [ALGORITHM] -->
+
+<details>
+<summary align="right"><a href="https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123460188.pdf">VoxelPose (ECCV'2020)</a></summary>
+
+```bibtex
+@inproceedings{tumultipose,
+  title={VoxelPose: Towards Multi-Camera 3D Human Pose Estimation in Wild Environment},
+  author={Tu, Hanyue and Wang, Chunyu and Zeng, Wenjun},
+  booktitle={ECCV},
+  year={2020}
+}
+```
+
+</details>
+
+<!-- [DATASET] -->
+
+<details>
+<summary align="right"><a href="http://campar.in.tum.de/pub/belagiannis2014cvpr/belagiannis2014cvpr.pdf">Campus (CVPR'2014)</a></summary>
+
+```bibtex
+@inproceedings {belagian14multi,
+    title = {{3D} Pictorial Structures for Multiple Human Pose Estimation},
+    author = {Belagiannis, Vasileios and Amin, Sikandar and Andriluka, Mykhaylo and Schiele, Bernt and Navab
+    Nassir and Ilic, Slobodan},
+    booktitle = {IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)},
+    year = {2014},
+    month = {June},
+    organization={IEEE}
+}
+```
+
+</details>
+
+Results on Campus dataset.
+
+| Arch | Actor 1 | Actor 2 | Actor 3 | Average| ckpt | log |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
+| [prn32_cpn80_res50](/configs/body/3d_kpt_mview_rgb_img/voxelpose/campus/voxelpose_prn32x32x32_cpn80x80x20_campus_cam3.py) | 97.76 | 93.92 | 98.48 | 96.72 | [ckpt](https://download.openmmlab.com/mmpose/body3d/voxelpose/voxelpose_prn32x32x32_cpn80x80x20_campus_cam3-3ecee30e_20220323.pth) | [log](https://download.openmmlab.com/mmpose/body3d/voxelpose/voxelpose_prn32x32x32_cpn80x80x20_campus_cam3_20220323.log.json) |
+| [prn64_cpn80_res50](/configs/body/3d_kpt_mview_rgb_img/voxelpose/campus/voxelpose_prn64x64x64_cpn80x80x20_campus_cam3.py) | 97.76 | 93.33 | 98.77 | 96.62| [ckpt](https://download.openmmlab.com/mmpose/body3d/voxelpose/voxelpose_prn64x64x64_cpn80x80x20_campus_cam3-d8decbf7_20220323.pth) | [log](https://download.openmmlab.com/mmpose/body3d/voxelpose/voxelpose_prn64x64x64_cpn80x80x20_campus_cam3_20220323.log.json) |
diff --git a/configs/body/3d_kpt_mview_rgb_img/voxelpose/campus/voxelpose_campus.yml b/configs/body/3d_kpt_mview_rgb_img/voxelpose/campus/voxelpose_campus.yml
@@ -0,0 +1,30 @@
+Collections:
+- Name: VoxelPose
+  Paper:
+    Title: 'VoxelPose: Towards Multi-Camera 3D Human Pose Estimation in Wild Environment'
+    URL: https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123460188.pdf
+  README: https://github.com/open-mmlab/mmpose/blob/master/docs/en/papers/algorithms/voxelpose.md
+Models:
+- Config: configs/body/3d_kpt_mview_rgb_img/voxelpose/campus/voxelpose_prn32x32x32_cpn80x80x20_campus_cam3.py
+  In Collection: VoxelPose
+  Metadata:
+    Architecture: &id001
+    - VoxelPose
+    Training Data: Campus
+  Name: voxelpose_voxelpose_prn32x32x32_cpn80x80x20_campus_cam3
+  Results:
+  - Dataset: Campus
+    Metrics: {}
+    Task: Body 3D Keypoint
+  Weights: https://download.openmmlab.com/mmpose/body3d/voxelpose/voxelpose_prn32x32x32_cpn80x80x20_campus_cam3-3ecee30e_20220323.pth
+- Config: configs/body/3d_kpt_mview_rgb_img/voxelpose/campus/voxelpose_prn64x64x64_cpn80x80x20_campus_cam3.py
+  In Collection: VoxelPose
+  Metadata:
+    Architecture: *id001
+    Training Data: Campus
+  Name: voxelpose_voxelpose_prn64x64x64_cpn80x80x20_campus_cam3
+  Results:
+  - Dataset: Campus
+    Metrics: {}
+    Task: Body 3D Keypoint
+  Weights: https://download.openmmlab.com/mmpose/body3d/voxelpose/voxelpose_prn64x64x64_cpn80x80x20_campus_cam3-d8decbf7_20220323.pth