diff --git a/README.md b/README.md index c277436ac4..53e103c835 100644 --- a/README.md +++ b/README.md @@ -82,7 +82,7 @@ a part of the OpenMMLab project developed by [MMLab](http://mmlab.ie.cuhk.edu.hk - **Support indoor/outdoor 3D detection out of box** It directly supports popular indoor and outdoor 3D detection datasets, including ScanNet, SUNRGB-D, Waymo, nuScenes, Lyft, and KITTI. - For nuScenes dataset, we also support [nuImages dataset](https://github.com/open-mmlab/mmdetection3d/tree/latest/configs/nuimages). + For nuScenes dataset, we also support [nuImages dataset](https://github.com/open-mmlab/mmdetection3d/tree/main/configs/nuimages). - **Natural integration with 2D detection** diff --git a/README_zh-CN.md b/README_zh-CN.md index 034c87e003..ee73c22967 100644 --- a/README_zh-CN.md +++ b/README_zh-CN.md @@ -78,7 +78,7 @@ MMDetection3D 是一个基于 PyTorch 的目标检测开源工具箱，下一代 - **支持户内/户外的数据集** 支持室内/室外的 3D 检测数据集，包括 ScanNet，SUNRGB-D，Waymo，nuScenes，Lyft，KITTI。 - 对于 nuScenes 数据集，我们也支持 [nuImages 数据集](https://github.com/open-mmlab/mmdetection3d/tree/latest/configs/nuimages)。 + 对于 nuScenes 数据集，我们也支持 [nuImages 数据集](https://github.com/open-mmlab/mmdetection3d/tree/main/configs/nuimages)。 - **与 2D 检测器的自然整合** diff --git a/configs/pointnet2/README.md b/configs/pointnet2/README.md index fc70a06ca9..955f2fbc50 100644 --- a/configs/pointnet2/README.md +++ b/configs/pointnet2/README.md @@ -35,7 +35,7 @@ We implement PointNet++ and provide the result and checkpoints on ScanNet and S3 - Since ScanNet dataset doesn't provide ground-truth labels for the test set, users can only evaluate test set performance by submitting to its online benchmark [website](http://kaldir.vc.in.tum.de/scannet_benchmark/). However, users are only allowed to submit once every two weeks. Therefore, we currently report val set mIoU. Test set performance may be added in the future. -- To generate submission file for ScanNet online benchmark, you need to modify the ScanNet dataset's [config](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/_base_/datasets/scannet_seg-3d-20class.py#L126). Change `ann_file=data_root + 'scannet_infos_val.pkl'` to `ann_file=data_root + 'scannet_infos_test.pkl'`, and then simply run: +- To generate submission file for ScanNet online benchmark, you need to modify the ScanNet dataset's [config](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/_base_/datasets/scannet-seg.py#L126). Change `ann_file=data_root + 'scannet_infos_val.pkl'` to `ann_file=data_root + 'scannet_infos_test.pkl'`, and then simply run: ```shell python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --format-only --options 'txt_prefix=exps/pointnet2_scannet_results' diff --git a/configs/second/README.md b/configs/second/README.md index 182a16eb63..e91c4bcab5 100644 --- a/configs/second/README.md +++ b/configs/second/README.md @@ -22,10 +22,10 @@ We implement SECOND and provide the results and checkpoints on KITTI dataset. | Backbone | Class | Lr schd | Mem (GB) | Inf time (fps) | mAP | Download | | :-----------------------------------------------------------------: | :-----: | :--------: | :------: | :------------: | :---: | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | -| [SECFPN](./second_hv_secfpn_8xb6-80e_kitti-3d-car.py) | Car | cyclic 80e | 5.4 | | 79.07 | [model](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/second/hv_second_secfpn_6x8_80e_kitti-3d-car/hv_second_secfpn_6x8_80e_kitti-3d-car_20200620_230238-393f000c.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/second/hv_second_secfpn_6x8_80e_kitti-3d-car/hv_second_secfpn_6x8_80e_kitti-3d-car_20200620_230238.log.json) | -| [SECFPN (FP16)](./hv_second_secfpn_fp16_6x8_80e_kitti-3d-car.py) | Car | cyclic 80e | 2.9 | | 78.72 | [model](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/fp16/hv_second_secfpn_fp16_6x8_80e_kitti-3d-car/hv_second_secfpn_fp16_6x8_80e_kitti-3d-car_20200924_211301-1f5ad833.pth)\| [log](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/fp16/hv_second_secfpn_fp16_6x8_80e_kitti-3d-car/hv_second_secfpn_fp16_6x8_80e_kitti-3d-car_20200924_211301.log.json) | -| [SECFPN](./second_hv_secfpn_8xb6-80e_kitti-3d-3class.py) | 3 Class | cyclic 80e | 5.4 | | 65.74 | [model](https://download.openmmlab.com/mmdetection3d/v1.0.0_models/second/hv_second_secfpn_6x8_80e_kitti-3d-3class/hv_second_secfpn_6x8_80e_kitti-3d-3class_20210831_022017-ae782e87.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.0.0_models/second/hv_second_secfpn_6x8_80e_kitti-3d-3class/hv_second_secfpn_6x8_80e_kitti-3d-3class_20210831_022017log.json) | -| [SECFPN (FP16)](./hv_second_secfpn_fp16_6x8_80e_kitti-3d-3class.py) | 3 Class | cyclic 80e | 2.9 | | 67.4 | [model](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/fp16/hv_second_secfpn_fp16_6x8_80e_kitti-3d-3class/hv_second_secfpn_fp16_6x8_80e_kitti-3d-3class_20200925_110059-05f67bdf.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/fp16/hv_second_secfpn_fp16_6x8_80e_kitti-3d-3class/hv_second_secfpn_fp16_6x8_80e_kitti-3d-3class_20200925_110059.log.json) | +| [SECFPN](./second_hv_secfpn_8xb6-80e_kitti-3d-car.py) | Car | cyclic 80e | 5.4 | | 78.2 | [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/second/second_hv_secfpn_8xb6-80e_kitti-3d-car/second_hv_secfpn_8xb6-80e_kitti-3d-car-75d9305e.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/second/second_hv_secfpn_8xb6-80e_kitti-3d-car/second_hv_secfpn_8xb6-80e_kitti-3d-car-20230420_191750.log) | +| [SECFPN (FP16)](./second_hv_secfpn_8xb6-amp-80e_kitti-3d-car.py) | Car | cyclic 80e | 2.9 | | 78.72 | [model](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/fp16/hv_second_secfpn_fp16_6x8_80e_kitti-3d-car/hv_second_secfpn_fp16_6x8_80e_kitti-3d-car_20200924_211301-1f5ad833.pth)\| [log](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/fp16/hv_second_secfpn_fp16_6x8_80e_kitti-3d-car/hv_second_secfpn_fp16_6x8_80e_kitti-3d-car_20200924_211301.log.json) | +| [SECFPN](./second_hv_secfpn_8xb6-80e_kitti-3d-3class.py) | 3 Class | cyclic 80e | 5.4 | | 65.3 | [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/second/second_hv_secfpn_8xb6-80e_kitti-3d-3class/second_hv_secfpn_8xb6-80e_kitti-3d-3class-b086d0a3.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/second/second_hv_secfpn_8xb6-80e_kitti-3d-3class/second_hv_secfpn_8xb6-80e_kitti-3d-3class-20230420_221130.log) | +| [SECFPN (FP16)](./second_hv_secfpn_8xb6-amp-80e_kitti-3d-3class.py) | 3 Class | cyclic 80e | 2.9 | | 67.4 | [model](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/fp16/hv_second_secfpn_fp16_6x8_80e_kitti-3d-3class/hv_second_secfpn_fp16_6x8_80e_kitti-3d-3class_20200925_110059-05f67bdf.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/fp16/hv_second_secfpn_fp16_6x8_80e_kitti-3d-3class/hv_second_secfpn_fp16_6x8_80e_kitti-3d-3class_20200925_110059.log.json) | ### Waymo @@ -38,7 +38,7 @@ We implement SECOND and provide the results and checkpoints on KITTI dataset. Note: -- See more details about metrics and data split on Waymo [HERE](https://github.com/open-mmlab/mmdetection3d/tree/master/configs/pointpillars). For implementation details, we basically follow the original settings. All of these results are achieved without bells-and-whistles, e.g. ensemble, multi-scale training and test augmentation. +- See more details about metrics and data split on Waymo [HERE](https://github.com/open-mmlab/mmdetection3d/tree/main/configs/pointpillars). For implementation details, we basically follow the original settings. All of these results are achieved without bells-and-whistles, e.g. ensemble, multi-scale training and test augmentation. - `FP16` means Mixed Precision (FP16) is adopted in training. ## Citation diff --git a/configs/second/metafile.yml b/configs/second/metafile.yml index f26f7aa311..097fb73f29 100644 --- a/configs/second/metafile.yml +++ b/configs/second/metafile.yml @@ -25,8 +25,8 @@ Models: - Task: 3D Object Detection Dataset: KITTI Metrics: - mAP: 79.07 - Weights: https://download.openmmlab.com/mmdetection3d/v0.1.0_models/second/hv_second_secfpn_6x8_80e_kitti-3d-car/hv_second_secfpn_6x8_80e_kitti-3d-car_20200620_230238-393f000c.pth + mAP: 78.2 + Weights: https://download.openmmlab.com/mmdetection3d/v1.1.0_models/second/second_hv_secfpn_8xb6-80e_kitti-3d-car/second_hv_secfpn_8xb6-80e_kitti-3d-car-75d9305e.pth - Name: second_hv_secfpn_8xb6-80e_kitti-3d-3class In Collection: SECOND @@ -39,8 +39,8 @@ Models: - Task: 3D Object Detection Dataset: KITTI Metrics: - mAP: 65.74 - Weights: https://download.openmmlab.com/mmdetection3d/v1.0.0_models/second/hv_second_secfpn_6x8_80e_kitti-3d-3class/hv_second_secfpn_6x8_80e_kitti-3d-3class_20210831_022017-ae782e87.pth + mAP: 65.3 + Weights: https://download.openmmlab.com/mmdetection3d/v1.1.0_models/second/second_hv_secfpn_8xb6-80e_kitti-3d-3class/second_hv_secfpn_8xb6-80e_kitti-3d-3class-b086d0a3.pth - Name: second_hv_secfpn_sbn-all_16xb2-2x_waymoD5-3d-3class In Collection: SECOND diff --git a/demo/data/nuscenes/n015-2018-07-24-11-22-45+0800.pkl b/demo/data/nuscenes/n015-2018-07-24-11-22-45+0800.pkl new file mode 100644 index 0000000000..95b51083a5 Binary files /dev/null and b/demo/data/nuscenes/n015-2018-07-24-11-22-45+0800.pkl differ diff --git a/demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__CAM_BACK__1532402927637525_mono3d.pkl b/demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__CAM_BACK__1532402927637525_mono3d.pkl deleted file mode 100644 index 020cdd3d29..0000000000 Binary files a/demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__CAM_BACK__1532402927637525_mono3d.pkl and /dev/null differ diff --git a/demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__LIDAR_TOP__1532402927647951.pcd.bin b/demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__LIDAR_TOP__1532402927647951.pcd.bin new file mode 100644 index 0000000000..b6388d5416 Binary files /dev/null and b/demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__LIDAR_TOP__1532402927647951.pcd.bin differ diff --git a/demo/data/sunrgbd/sunrgbd_000017.bin b/demo/data/sunrgbd/000017.bin similarity index 100% rename from demo/data/sunrgbd/sunrgbd_000017.bin rename to demo/data/sunrgbd/000017.bin diff --git a/demo/data/sunrgbd/sunrgbd_000017.jpg b/demo/data/sunrgbd/000017.jpg similarity index 100% rename from demo/data/sunrgbd/sunrgbd_000017.jpg rename to demo/data/sunrgbd/000017.jpg diff --git a/demo/data/sunrgbd/sunrgbd_000017_infos.pkl b/demo/data/sunrgbd/sunrgbd_000017_infos.pkl index 6557aeb18d..f222402bb6 100644 Binary files a/demo/data/sunrgbd/sunrgbd_000017_infos.pkl and b/demo/data/sunrgbd/sunrgbd_000017_infos.pkl differ diff --git a/docs/en/advanced_guides/datasets/scannet.md b/docs/en/advanced_guides/datasets/scannet.md index d17b72ff3b..dc6a1b9bdd 100644 --- a/docs/en/advanced_guides/datasets/scannet.md +++ b/docs/en/advanced_guides/datasets/scannet.md @@ -337,12 +337,12 @@ train_pipeline = [ ## Testing and Making a Submission By default, our codebase evaluates semantic segmentation results on the validation set. -If you would like to test the model performance on the online benchmark, add `--format-only` flag in the evaluation script and change `ann_file=data_root + 'scannet_infos_val.pkl'` to `ann_file=data_root + 'scannet_infos_test.pkl'` in the ScanNet dataset's [config](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/_base_/datasets/scannet_seg-3d-20class.py#L126). Remember to specify the `txt_prefix` as the directory to save the testing results. +If you would like to test the model performance on the online benchmark, add `--format-only` flag in the evaluation script and change `ann_file=data_root + 'scannet_infos_val.pkl'` to `ann_file=data_root + 'scannet_infos_test.pkl'` in the ScanNet dataset's [config](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/_base_/datasets/scannet-seg.py#L126). Remember to specify the `txt_prefix` as the directory to save the testing results. Taking PointNet++ (SSG) on ScanNet for example, the following command can be used to do inference on test set: ``` -./tools/dist_test.sh configs/pointnet2/pointnet2_ssg_16x2_cosine_200e_scannet_seg-3d-20class.py \ +./tools/dist_test.sh configs/pointnet2/pointnet2_ssg_16x2_cosine_200e_scannet-seg.py \ work_dirs/pointnet2_ssg/latest.pth --format-only \ --eval-options txt_prefix=work_dirs/pointnet2_ssg/test_submission ``` diff --git a/docs/en/advanced_guides/supported_tasks/lidar_sem_seg3d.md b/docs/en/advanced_guides/supported_tasks/lidar_sem_seg3d.md index 01e62bd408..521e3b20fe 100644 --- a/docs/en/advanced_guides/supported_tasks/lidar_sem_seg3d.md +++ b/docs/en/advanced_guides/supported_tasks/lidar_sem_seg3d.md @@ -74,14 +74,14 @@ The evaluation results will be printed in the command like: In addition, you can also evaluate a specific model checkpoint after training is finished. Simply run scripts like the following: ``` -./tools/dist_test.sh configs/pointnet2/pointnet2_ssg_16x2_cosine_200e_scannet_seg-3d-20class.py work_dirs/pointnet2_ssg/latest.pth 8 +./tools/dist_test.sh configs/pointnet2/pointnet2_ssg_16x2_cosine_200e_scannet-seg.py work_dirs/pointnet2_ssg/latest.pth 8 ``` ## Testing and Making a Submission If you would like to only conduct inference or test the model performance on the online benchmark, you should change `ann_file='scannet_infos_val.pkl'` to `ann_file='scannet_infos_test.pkl'` in the -ScanNet dataset's [config](https://github.com/open-mmlab/mmdetection3d/blob/dev-1.x/configs/_base_/datasets/scannet_seg-3d-20class.py#L129). Remember to +ScanNet dataset's [config](https://github.com/open-mmlab/mmdetection3d/blob/dev-1.x/configs/_base_/datasets/scannet-seg.py#L129). Remember to specify the `submission_prefix` in the `test_evaluator`, e.g., adding `test_evaluator = dict(type='SegMetric', submission_prefix=work_dirs/pointnet2_ssg/test_submission`) or just add `--cfg-options test_evaluator.submission_prefix=work_dirs/pointnet2_ssg/test_submission` in the end of command. After generating the results, you can basically compress the folder and upload to the [ScanNet evaluation server](http://kaldir.vc.in.tum.de/scannet_benchmark/semantic_label_3d). diff --git a/docs/en/model_zoo.md b/docs/en/model_zoo.md index a8956d4af7..a16ee554cb 100644 --- a/docs/en/model_zoo.md +++ b/docs/en/model_zoo.md @@ -10,75 +10,75 @@ ### SECOND -Please refer to [SECOND](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/second) for details. We provide SECOND baselines on KITTI and Waymo datasets. +Please refer to [SECOND](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/second) for details. We provide SECOND baselines on KITTI and Waymo datasets. ### PointPillars -Please refer to [PointPillars](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/pointpillars) for details. We provide pointpillars baselines on KITTI, nuScenes, Lyft, and Waymo datasets. +Please refer to [PointPillars](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/pointpillars) for details. We provide pointpillars baselines on KITTI, nuScenes, Lyft, and Waymo datasets. ### Part-A2 -Please refer to [Part-A2](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/parta2) for details. +Please refer to [Part-A2](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/parta2) for details. ### VoteNet -Please refer to [VoteNet](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/votenet) for details. We provide VoteNet baselines on ScanNet and SUNRGBD datasets. +Please refer to [VoteNet](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/votenet) for details. We provide VoteNet baselines on ScanNet and SUNRGBD datasets. ### Dynamic Voxelization -Please refer to [Dynamic Voxelization](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/dynamic_voxelization) for details. +Please refer to [Dynamic Voxelization](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/dynamic_voxelization) for details. ### MVXNet -Please refer to [MVXNet](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/mvxnet) for details. +Please refer to [MVXNet](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/mvxnet) for details. ### RegNetX -Please refer to [RegNet](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/regnet) for details. We provide pointpillars baselines with RegNetX backbones on nuScenes and Lyft datasets currently. +Please refer to [RegNet](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/regnet) for details. We provide pointpillars baselines with RegNetX backbones on nuScenes and Lyft datasets currently. ### nuImages -We also support baseline models on [nuImages dataset](https://www.nuscenes.org/nuimages). Please refer to [nuImages](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/nuimages) for details. We report Mask R-CNN, Cascade Mask R-CNN and HTC results currently. +We also support baseline models on [nuImages dataset](https://www.nuscenes.org/nuimages). Please refer to [nuImages](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/nuimages) for details. We report Mask R-CNN, Cascade Mask R-CNN and HTC results currently. ### H3DNet -Please refer to [H3DNet](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/h3dnet) for details. +Please refer to [H3DNet](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/h3dnet) for details. ### 3DSSD -Please refer to [3DSSD](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/3dssd) for details. +Please refer to [3DSSD](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/3dssd) for details. ### CenterPoint -Please refer to [CenterPoint](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/centerpoint) for details. +Please refer to [CenterPoint](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/centerpoint) for details. ### SSN -Please refer to [SSN](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/ssn) for details. We provide pointpillars with shape-aware grouping heads used in SSN on the nuScenes and Lyft datasets currently. +Please refer to [SSN](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/ssn) for details. We provide pointpillars with shape-aware grouping heads used in SSN on the nuScenes and Lyft datasets currently. ### ImVoteNet -Please refer to [ImVoteNet](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/imvotenet) for details. We provide ImVoteNet baselines on SUNRGBD dataset. +Please refer to [ImVoteNet](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/imvotenet) for details. We provide ImVoteNet baselines on SUNRGBD dataset. ### FCOS3D -Please refer to [FCOS3D](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/fcos3d) for details. We provide FCOS3D baselines on the nuScenes dataset. +Please refer to [FCOS3D](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/fcos3d) for details. We provide FCOS3D baselines on the nuScenes dataset. ### PointNet++ -Please refer to [PointNet++](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/pointnet2) for details. We provide PointNet++ baselines on ScanNet and S3DIS datasets. +Please refer to [PointNet++](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/pointnet2) for details. We provide PointNet++ baselines on ScanNet and S3DIS datasets. ### Group-Free-3D -Please refer to [Group-Free-3D](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/groupfree3d) for details. We provide Group-Free-3D baselines on ScanNet dataset. +Please refer to [Group-Free-3D](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/groupfree3d) for details. We provide Group-Free-3D baselines on ScanNet dataset. ### ImVoxelNet -Please refer to [ImVoxelNet](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/imvoxelnet) for details. We provide ImVoxelNet baselines on KITTI dataset. +Please refer to [ImVoxelNet](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/imvoxelnet) for details. We provide ImVoxelNet baselines on KITTI dataset. ### PAConv -Please refer to [PAConv](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/paconv) for details. We provide PAConv baselines on S3DIS dataset. +Please refer to [PAConv](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/paconv) for details. We provide PAConv baselines on S3DIS dataset. ### DGCNN @@ -102,11 +102,11 @@ Please refer to [MonoFlex](https://github.com/open-mmlab/mmdetection3d/tree/v1.0 ### SA-SSD -Please refer to [SA-SSD](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/sassd) for details. We provide SA-SSD baselines on the KITTI dataset. +Please refer to [SA-SSD](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/sassd) for details. We provide SA-SSD baselines on the KITTI dataset. ### FCAF3D -Please refer to [FCAF3D](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/fcaf3d) for details. We provide FCAF3D baselines on the ScanNet, S3DIS, and SUN RGB-D datasets. +Please refer to [FCAF3D](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/fcaf3d) for details. We provide FCAF3D baselines on the ScanNet, S3DIS, and SUN RGB-D datasets. ### PV-RCNN diff --git a/docs/en/notes/benchmarks.md b/docs/en/notes/benchmarks.md index 4db2c39546..c84ffc8317 100644 --- a/docs/en/notes/benchmarks.md +++ b/docs/en/notes/benchmarks.md @@ -26,7 +26,7 @@ We compare the training speed (samples/s) with other codebases if they implement ### Modification for Calculating Speed -- __MMDetection3D__: We try to use as similar settings as those of other codebases as possible using [benchmark configs](https://github.com/open-mmlab/MMDetection3D/blob/master/configs/benchmark). +- __MMDetection3D__: We try to use as similar settings as those of other codebases as possible using [benchmark configs](https://github.com/open-mmlab/MMDetection3D/blob/main/configs/benchmark). - __Det3D__: For comparison with Det3D, we use the commit [519251e](https://github.com/poodarchu/Det3D/tree/519251e72a5c1fdd58972eabeac67808676b9bb7). diff --git a/docs/en/user_guides/inference.md b/docs/en/user_guides/inference.md index f692097e26..cca03c8ce5 100644 --- a/docs/en/user_guides/inference.md +++ b/docs/en/user_guides/inference.md @@ -18,20 +18,18 @@ python demo/pcd_demo.py ${PCD_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device The visualization results including a point cloud and predicted 3D bounding boxes will be saved in `${OUT_DIR}/PCD_NAME`, which you can open using [MeshLab](http://www.meshlab.net/). Note that if you set the flag `--show`, the prediction result will be displayed online using [Open3D](http://www.open3d.org/). -Example on KITTI data using [SECOND](https://github.com/open-mmlab/mmdetection3d/tree/master/configs/second) model: +Example on KITTI data using [PointPillars model](https://download.openmmlab.com/mmdetection3d/v1.0.0_models/pointpillars/hv_pointpillars_secfpn_6x8_160e_kitti-3d-car/hv_pointpillars_secfpn_6x8_160e_kitti-3d-car_20220331_134606-d42d15ed.pth): ```shell -python demo/pcd_demo.py demo/data/kitti/000008.bin configs/second/second_hv-secfpn_8xb6-80e_kitti-3d-car.py checkpoints/second_hv-secfpn_8xb6-80e_kitti-3d-car_20200620_230238-393f000c.pth +python demo/pcd_demo.py demo/data/kitti/000008.bin configs/pointpillars/pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car.py ${CHECKPOINT_FILE} --show ``` -Example on SUN RGB-D data using [VoteNet](https://github.com/open-mmlab/mmdetection3d/tree/master/configs/votenet) model: +Example on SUN RGB-D data using [VoteNet model](https://download.openmmlab.com/mmdetection3d/v1.0.0_models/votenet/votenet_16x8_sunrgbd-3d-10class/votenet_16x8_sunrgbd-3d-10class_20210820_162823-bf11f014.pth): ```shell -python demo/pcd_demo.py demo/data/sunrgbd/sunrgbd_000017.bin configs/votenet/votenet_8xb16_sunrgbd-3d.py checkpoints/votenet_8xb16_sunrgbd-3d_20200620_230238-4483c0c0.pth +python demo/pcd_demo.py demo/data/sunrgbd/sunrgbd_000017.bin configs/votenet/votenet_8xb16_sunrgbd-3d.py ${CHECKPOINT_FILE} --show ``` -Remember to convert the VoteNet checkpoint if you are using mmdetection3d version >= 0.6.0. See its [README](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/votenet/README.md/) for detailed instructions on how to convert the checkpoint. - #### Monocular 3D demo To test a monocular 3D detector on image data, simply run: @@ -42,13 +40,21 @@ python demo/mono_det_demo.py ${IMAGE_FILE} ${ANNOTATION_FILE} ${CONFIG_FILE} ${C where the `ANNOTATION_FILE` should provide the 3D to 2D projection matrix (camera intrinsic matrix), and `CAM_TYPE` should be specified according to dataset. For example, if you want to inference on the front camera image, the `CAM_TYPE` should be set as `CAM_2` for KITTI, and `CAM_FRONT` for nuScenes. By specifying `CAM_TYPE`, you can even infer on any camera images for datasets with multi-view cameras, such as nuScenes and Waymo. `SCORE-THR` is the 3D bbox threshold while visualization. The visualization results including an image and its predicted 3D bounding boxes projected on the image will be saved in `${OUT_DIR}/IMG_NAME`. -Example on nuScenes data using [FCOS3D](https://github.com/open-mmlab/mmdetection3d/tree/master/configs/fcos3d) model: +Example on KITTI data using [PGD model](https://download.openmmlab.com/mmdetection3d/v1.0.0_models/pgd/pgd_r101_caffe_fpn_gn-head_3x4_4x_kitti-mono3d/pgd_r101_caffe_fpn_gn-head_3x4_4x_kitti-mono3d_20211022_102608-8a97533b.pth): + +```shell +python demo/mono_det_demo.py demo/data/kitti/000008.png demo/data/kitti/000008.pkl configs/pgd/pgd_r101-caffe_fpn_head-gn_4xb3-4x_kitti-mono3d.py ${CHECKPOINT_FILE} --show --cam-type CAM2 --score-thr 8 +``` + +**Note**: For PGD, the prediction score is not among (0, 1). + +Example on nuScenes data using [FCOS3D model](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/fcos3d/fcos3d_r101_caffe_fpn_gn-head_dcn_2x8_1x_nus-mono3d_finetune/fcos3d_r101_caffe_fpn_gn-head_dcn_2x8_1x_nus-mono3d_finetune_20210717_095645-8d806dc2.pth): ```shell -python demo/mono_det_demo.py demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__CAM_BACK__1532402927637525.jpg demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__CAM_BACK__1532402927637525.pkl configs/fcos3d/fcos3d_r101-caffe-dcn-fpn-head-gn_8xb2-1x_nus-mono3d_finetune.py checkpoints/fcos3d_r101-caffe-dcn-fpn-head-gn_8xb2-1x_nus-mono3d_finetune_20210717_095645-8d806dc2.pth +python demo/mono_det_demo.py demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__CAM_BACK__1532402927637525.jpg demo/data/nuscenes/n015-2018-07-24-11-22-45+0800.pkl configs/fcos3d/fcos3d_r101-caffe-dcn_fpn_head-gn_8xb2-1x_nus-mono3d_finetune.py ${CHECKPOINT_FILE} --show --cam-type CAM_BACK ``` -Note that when visualizing results of monocular 3D detection for flipped images, the camera intrinsic matrix should also be modified accordingly. See more details and examples in PR [#744](https://github.com/open-mmlab/mmdetection3d/pull/744). +**Note** that when visualizing results of monocular 3D detection for flipped images, the camera intrinsic matrix should also be modified accordingly. See more details and examples in PR [#744](https://github.com/open-mmlab/mmdetection3d/pull/744). #### Multi-modality demo @@ -60,16 +66,16 @@ python demo/multi_modality_demo.py ${PCD_FILE} ${IMAGE_FILE} ${ANNOTATION_FILE} where the `ANNOTATION_FILE` should provide the 3D to 2D projection matrix. The visualization results including a point cloud, an image, predicted 3D bounding boxes and their projection on the image will be saved in `${OUT_DIR}/PCD_NAME`. -Example on KITTI data using [MVX-Net](https://github.com/open-mmlab/mmdetection3d/tree/master/configs/mvxnet) model: +Example on KITTI data using [MVX-Net](https://github.com/open-mmlab/mmdetection3d/tree/main/configs/mvxnet) model: ```shell -python demo/multi_modality_demo.py demo/data/kitti/000008.bin demo/data/kitti/000008.png demo/data/kitti/000008.pkl configs/mvxnet/mvx_fpn-dv-second-secfpn_8xb2-80e_kitti-3d-3class.py checkpoints/mvx_fpn-dv-second-secfpn_8xb2-80e_kitti-3d-3class_20200621_003904-10140f2d.pth +python demo/multi_modality_demo.py demo/data/kitti/000008.bin demo/data/kitti/000008.png demo/data/kitti/000008.pkl configs/mvxnet/mvxnet_fpn_dv_second_secfpn_8xb2-80e_kitti-3d-3class.py ${CHECKPOINT_FILE} --cam-type CAM2 --show ``` -Example on SUN RGB-D data using [ImVoteNet](https://github.com/open-mmlab/mmdetection3d/tree/master/configs/imvotenet) model: +Example on SUN RGB-D data using [ImVoteNet model](https://download.openmmlab.com/mmdetection3d/v1.0.0_models/imvotenet/imvotenet_stage2_16x8_sunrgbd-3d-10class/imvotenet_stage2_16x8_sunrgbd-3d-10class_20210819_192851-1bcd1b97.pth): ```shell -python demo/multi_modality_demo.py demo/data/sunrgbd/sunrgbd_000017.bin demo/data/sunrgbd/sunrgbd_000017.jpg demo/data/sunrgbd/sunrgbd_000017_infos.pkl configs/imvotenet/imvotenet_stage2_8xb16_sunrgbd.py checkpoints/imvotenet_stage2_8xb16_sunrgbd_20210323_184021-d44dcb66.pth +python demo/multi_modality_demo.py demo/data/sunrgbd/000017.bin demo/data/sunrgbd/000017.jpg demo/data/sunrgbd/sunrgbd_000017_infos.pkl configs/imvotenet/imvotenet_stage2_8xb16_sunrgbd-3d.py ${CHECKPOINT_FILE} --cam-type CAM0 --show --score-thr 0.6 ``` ### 3D Segmentation @@ -82,8 +88,8 @@ python demo/pcd_seg_demo.py ${PCD_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--dev The visualization results including a point cloud and its predicted 3D segmentation mask will be saved in `${OUT_DIR}/PCD_NAME`. -Example on ScanNet data using [PointNet++ (SSG)](https://github.com/open-mmlab/mmdetection3d/tree/master/configs/pointnet2) model: +Example on ScanNet data using [PointNet++ (SSG) model](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/pointnet2/pointnet2_ssg_16x2_cosine_200e_scannet_seg-3d-20class/pointnet2_ssg_16x2_cosine_200e_scannet_seg-3d-20class_20210514_143644-ee73704a.pth): ```shell -python demo/pc_seg_demo.py demo/data/scannet/scene0000_00.bin configs/pointnet2/pointnet2_ssg_2xb16-cosine-200e_scannet-seg.py checkpoints/pointnet2_ssg_2xb16-cosine-200e_scannet-seg_20210514_143644-ee73704a.pth +python demo/pcd_seg_demo.py demo/data/scannet/scene0000_00.bin configs/pointnet2/pointnet2_ssg_2xb16-cosine-200e_scannet-seg.py ${CHECKPOINT_FILE} --show ``` diff --git a/docs/en/user_guides/useful_tools.md b/docs/en/user_guides/useful_tools.md index aa2e347373..ab73ae3702 100644 --- a/docs/en/user_guides/useful_tools.md +++ b/docs/en/user_guides/useful_tools.md @@ -202,7 +202,7 @@ python -u tools/dataset_converters/nuimage_converter.py --data-root ${DATA_ROOT} - `--nproc`: number of workers for data preparation, defaults to `4`. Larger number could reduce the preparation time as images are processed in parallel. - `--extra-tag`: extra tag of the annotations, defaults to `nuimages`. This can be used to separate different annotations processed in different time for study. -More details could be referred to the [doc](https://mmdetection3d.readthedocs.io/en/latest/data_preparation.html) for dataset preparation and [README](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/nuimages/README.md/) for nuImages dataset. +More details could be referred to the [doc](https://mmdetection3d.readthedocs.io/en/latest/data_preparation.html) for dataset preparation and [README](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/nuimages/README.md/) for nuImages dataset.   diff --git a/docs/en/user_guides/visualization.md b/docs/en/user_guides/visualization.md index 397e5e3355..9aa0c69fb3 100644 --- a/docs/en/user_guides/visualization.md +++ b/docs/en/user_guides/visualization.md @@ -190,7 +190,7 @@ python tools/misc/browse_dataset.py configs/mvxnet/dv_mvx-fpn_second_secfpn_adam You can simply browse different datasets using different configs, e.g. visualizing the ScanNet dataset in 3D semantic segmentation task: ```shell -python tools/misc/browse_dataset.py configs/_base_/datasets/scannet_seg-3d-20class.py --task lidar_seg --output-dir ${OUTPUT_DIR} --online +python tools/misc/browse_dataset.py configs/_base_/datasets/scannet-seg.py --task lidar_seg --output-dir ${OUTPUT_DIR} --online ``` ![](../../../resources/browse_dataset_seg.png) diff --git a/docs/zh_cn/advanced_guides/datasets/scannet.md b/docs/zh_cn/advanced_guides/datasets/scannet.md index c2e2c6bcfd..bbba255a2c 100644 --- a/docs/zh_cn/advanced_guides/datasets/scannet.md +++ b/docs/zh_cn/advanced_guides/datasets/scannet.md @@ -339,14 +339,14 @@ train_pipeline = [ 默认情况下，MMDet3D 的代码是在训练集上进行模型训练，然后在验证集上进行模型测试。 -如果你也想在在线基准上测试模型的性能（仅支持语义分割），请在测试命令中加上 `--format-only` 的标记，同时也要将 ScanNet 数据集[配置文件](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/_base_/datasets/scannet_seg-3d-20class.py#L126)中的 `ann_file=data_root + 'scannet_infos_val.pkl'` 改成 `ann_file=data_root + 'scannet_infos_test.pkl'`。 +如果你也想在在线基准上测试模型的性能（仅支持语义分割），请在测试命令中加上 `--format-only` 的标记，同时也要将 ScanNet 数据集[配置文件](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/_base_/datasets/scannet-seg.py#L126)中的 `ann_file=data_root + 'scannet_infos_val.pkl'` 改成 `ann_file=data_root + 'scannet_infos_test.pkl'`。 请记得通过 `txt_prefix` 来指定想要保存测试结果的文件夹名称。 以 PointNet++ (SSG) 在 ScanNet 上的测试为例，你可以运行以下命令来完成测试结果的保存： ``` -./tools/dist_test.sh configs/pointnet2/pointnet2_ssg_16x2_cosine_200e_scannet_seg-3d-20class.py \ +./tools/dist_test.sh configs/pointnet2/pointnet2_ssg_16x2_cosine_200e_scannet-seg.py \ work_dirs/pointnet2_ssg/latest.pth --format-only \ --eval-options txt_prefix=work_dirs/pointnet2_ssg/test_submission ``` diff --git a/docs/zh_cn/advanced_guides/supported_tasks/lidar_sem_seg3d.md b/docs/zh_cn/advanced_guides/supported_tasks/lidar_sem_seg3d.md index 0e24b55d20..d35636ad7a 100644 --- a/docs/zh_cn/advanced_guides/supported_tasks/lidar_sem_seg3d.md +++ b/docs/zh_cn/advanced_guides/supported_tasks/lidar_sem_seg3d.md @@ -65,13 +65,13 @@ mmdetection3d 此外，在训练完成后你也可以评估特定的模型权重文件。你可以简单地执行以下脚本： ``` -./tools/dist_test.sh configs/pointnet2/pointnet2_ssg_16x2_cosine_200e_scannet_seg-3d-20class.py work_dirs/pointnet2_ssg/latest.pth 8 +./tools/dist_test.sh configs/pointnet2/pointnet2_ssg_16x2_cosine_200e_scannet-seg.py work_dirs/pointnet2_ssg/latest.pth 8 ``` ## 测试与提交 如果你只想在在线基准上进行推理或测试模型性能，你需要在配置文件中的 `test_evalutor` 字段增加 `submission_prefix`， 例如配置文件增加 `test_evaluator = dict(type='SegMetric',submission_prefix=work_dirs/pointnet2_ssg/test_submission`)。 -并将 ScanNet 数据集[配置文件](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/_base_/datasets/scannet_seg-3d-20class.py#L129)中的 `ann_file=scannet_infos_val.pkl` 变成 `ann_file=scannet_infos_test.pkl`。在生成结果后，你可以压缩文件夹并上传至 [ScanNet 评估服务器](http://kaldir.vc.in.tum.de/scannet_benchmark/semantic_label_3d)上。 +并将 ScanNet 数据集[配置文件](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/_base_/datasets/scannet-seg.py#L129)中的 `ann_file=scannet_infos_val.pkl` 变成 `ann_file=scannet_infos_test.pkl`。在生成结果后，你可以压缩文件夹并上传至 [ScanNet 评估服务器](http://kaldir.vc.in.tum.de/scannet_benchmark/semantic_label_3d)上。 ## 定性评估 diff --git a/docs/zh_cn/advanced_guides/supported_tasks/vision_det3d.md b/docs/zh_cn/advanced_guides/supported_tasks/vision_det3d.md index 18c546e999..ff5917cae8 100644 --- a/docs/zh_cn/advanced_guides/supported_tasks/vision_det3d.md +++ b/docs/zh_cn/advanced_guides/supported_tasks/vision_det3d.md @@ -58,7 +58,7 @@ mmdetection3d ./tools/dist_train.sh fcos3d_r101_caffe_fpn_gn-head_dcn_2x8_1x_nus-mono3d_finetune.py 8 ``` -通过先前的脚本训练好一个基准模型后，请记得相应的修改[此处](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/fcos3d/fcos3d_r101_caffe_fpn_gn-head_dcn_2x8_1x_nus-mono3d_finetune.py#L8)的路径。 +通过先前的脚本训练好一个基准模型后，请记得相应的修改[此处](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/fcos3d/fcos3d_r101_caffe_fpn_gn-head_dcn_2x8_1x_nus-mono3d_finetune.py#L8)的路径。 ## 定量评估 @@ -101,7 +101,7 @@ barrier 0.466 0.581 0.269 0.169 nan nan ## 测试与提交 -如果你只想在在线基准上进行推理或测试模型性能，你需要将之前评估脚本中的 `--eval mAP` 替换成 `--format-only`，并在需要的情况下指定 `jsonfile_prefix`，例如，添加选项 `--eval-options jsonfile_prefix=work_dirs/fcos3d/test_submission`。请确保配置文件中的[测试信息](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/_base_/datasets/nus-mono3d.py#L93)由验证集相应地改为测试集。 +如果你只想在在线基准上进行推理或测试模型性能，你需要将之前评估脚本中的 `--eval mAP` 替换成 `--format-only`，并在需要的情况下指定 `jsonfile_prefix`，例如，添加选项 `--eval-options jsonfile_prefix=work_dirs/fcos3d/test_submission`。请确保配置文件中的[测试信息](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/_base_/datasets/nus-mono3d.py#L93)由验证集相应地改为测试集。 在生成结果后，你可以压缩文件夹并上传至 nuScenes 3D 检测挑战的 evalAI 评估服务器上。 diff --git a/docs/zh_cn/model_zoo.md b/docs/zh_cn/model_zoo.md index 2a443f3d15..56739f8015 100644 --- a/docs/zh_cn/model_zoo.md +++ b/docs/zh_cn/model_zoo.md @@ -10,104 +10,104 @@ ### SECOND -请参考 [SECOND](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/second) 获取更多的细节，我们在 KITTI 和 Waymo 数据集上都给出了相应的基准结果。 +请参考 [SECOND](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/second) 获取更多的细节，我们在 KITTI 和 Waymo 数据集上都给出了相应的基准结果。 ### PointPillars -请参考 [PointPillars](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/pointpillars) 获取更多细节，我们在 KITTI 、nuScenes 、Lyft 、Waymo 数据集上给出了相应的基准结果。 +请参考 [PointPillars](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/pointpillars) 获取更多细节，我们在 KITTI 、nuScenes 、Lyft 、Waymo 数据集上给出了相应的基准结果。 ### Part-A2 -请参考 [Part-A2](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/parta2) 获取更多细节。 +请参考 [Part-A2](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/parta2) 获取更多细节。 ### VoteNet -请参考 [VoteNet](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/votenet) 获取更多细节，我们在 ScanNet 和 SUNRGBD 数据集上给出了相应的基准结果。 +请参考 [VoteNet](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/votenet) 获取更多细节，我们在 ScanNet 和 SUNRGBD 数据集上给出了相应的基准结果。 ### Dynamic Voxelization -请参考 [Dynamic Voxelization](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/dynamic_voxelization) 获取更多细节。 +请参考 [Dynamic Voxelization](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/dynamic_voxelization) 获取更多细节。 ### MVXNet -请参考 [MVXNet](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/mvxnet) 获取更多细节。 +请参考 [MVXNet](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/mvxnet) 获取更多细节。 ### RegNetX -请参考 [RegNet](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/regnet) 获取更多细节，我们将 pointpillars 的主干网络替换成 RegNetX，并在 nuScenes 和 Lyft 数据集上给出了相应的基准结果。 +请参考 [RegNet](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/regnet) 获取更多细节，我们将 pointpillars 的主干网络替换成 RegNetX，并在 nuScenes 和 Lyft 数据集上给出了相应的基准结果。 ### nuImages -我们在 [nuImages 数据集](https://www.nuscenes.org/nuimages) 上也提供基准模型，请参考 [nuImages](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/nuimages) 获取更多细节，我们在该数据集上提供 Mask R-CNN ， Cascade Mask R-CNN 和 HTC 的结果。 +我们在 [nuImages 数据集](https://www.nuscenes.org/nuimages) 上也提供基准模型，请参考 [nuImages](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/nuimages) 获取更多细节，我们在该数据集上提供 Mask R-CNN ， Cascade Mask R-CNN 和 HTC 的结果。 ### H3DNet -请参考 [H3DNet](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/h3dnet) 获取更多细节。 +请参考 [H3DNet](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/h3dnet) 获取更多细节。 ### 3DSSD -请参考 [3DSSD](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/3dssd) 获取更多细节。 +请参考 [3DSSD](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/3dssd) 获取更多细节。 ### CenterPoint -请参考 [CenterPoint](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/centerpoint) 获取更多细节。 +请参考 [CenterPoint](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/centerpoint) 获取更多细节。 ### SSN -请参考 [SSN](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/ssn) 获取更多细节，我们将 pointpillars 中的检测头替换成 SSN 模型中所使用的 ‘shape-aware grouping heads’，并在 nuScenes 和 Lyft 数据集上给出了相应的基准结果。 +请参考 [SSN](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/ssn) 获取更多细节，我们将 pointpillars 中的检测头替换成 SSN 模型中所使用的 ‘shape-aware grouping heads’，并在 nuScenes 和 Lyft 数据集上给出了相应的基准结果。 ### ImVoteNet -请参考 [ImVoteNet](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/imvotenet) 获取更多细节，我们在 SUNRGBD 数据集上给出了相应的结果。 +请参考 [ImVoteNet](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/imvotenet) 获取更多细节，我们在 SUNRGBD 数据集上给出了相应的结果。 ### FCOS3D -请参考 [FCOS3D](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/fcos3d) 获取更多细节，我们在 nuScenes 数据集上给出了相应的结果。 +请参考 [FCOS3D](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/fcos3d) 获取更多细节，我们在 nuScenes 数据集上给出了相应的结果。 ### PointNet++ -请参考 [PointNet++](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/pointnet2) 获取更多细节，我们在 ScanNet 和 S3DIS 数据集上给出了相应的结果。 +请参考 [PointNet++](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/pointnet2) 获取更多细节，我们在 ScanNet 和 S3DIS 数据集上给出了相应的结果。 ### Group-Free-3D -请参考 [Group-Free-3D](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/groupfree3d) 获取更多细节，我们在 ScanNet 数据集上给出了相应的结果。 +请参考 [Group-Free-3D](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/groupfree3d) 获取更多细节，我们在 ScanNet 数据集上给出了相应的结果。 ### ImVoxelNet -请参考 [ImVoxelNet](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/imvoxelnet) 获取更多细节，我们在 KITTI 数据集上给出了相应的结果。 +请参考 [ImVoxelNet](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/imvoxelnet) 获取更多细节，我们在 KITTI 数据集上给出了相应的结果。 ### PAConv -请参考 [PAConv](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/paconv) 获取更多细节，我们在 S3DIS 数据集上给出了相应的结果。 +请参考 [PAConv](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/paconv) 获取更多细节，我们在 S3DIS 数据集上给出了相应的结果。 ### DGCNN -请参考 [DGCNN](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/dgcnn) 获取更多细节，我们在 S3DIS 数据集上给出了相应的结果。 +请参考 [DGCNN](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/dgcnn) 获取更多细节，我们在 S3DIS 数据集上给出了相应的结果。 ### SMOKE -请参考 [SMOKE](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/smoke) 获取更多细节，我们在 KITTI 数据集上给出了相应的结果。 +请参考 [SMOKE](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/smoke) 获取更多细节，我们在 KITTI 数据集上给出了相应的结果。 ### PGD -请参考 [PGD](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/pgd) 获取更多细节，我们在 KITTI 和 nuScenes 数据集上给出了相应的结果。 +请参考 [PGD](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/pgd) 获取更多细节，我们在 KITTI 和 nuScenes 数据集上给出了相应的结果。 ### PointRCNN -请参考 [PointRCNN](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/point_rcnn) 获取更多细节，我们在 KITTI 数据集上给出了相应的结果。 +请参考 [PointRCNN](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/point_rcnn) 获取更多细节，我们在 KITTI 数据集上给出了相应的结果。 ### MonoFlex -请参考 [MonoFlex](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/monoflex) 获取更多细节，我们在 KITTI 数据集上给出了相应的结果。 +请参考 [MonoFlex](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/monoflex) 获取更多细节，我们在 KITTI 数据集上给出了相应的结果。 ### SA-SSD -请参考 [SA-SSD](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/sassd) 获取更多的细节，我们在 KITTI 数据集上给出了相应的基准结果。 +请参考 [SA-SSD](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/sassd) 获取更多的细节，我们在 KITTI 数据集上给出了相应的基准结果。 ### FCAF3D -请参考 [FCAF3D](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/fcaf3d) 获取更多的细节，我们在 ScanNet, S3DIS 和 SUN RGB-D 数据集上给出了相应的基准结果。 +请参考 [FCAF3D](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/fcaf3d) 获取更多的细节，我们在 ScanNet, S3DIS 和 SUN RGB-D 数据集上给出了相应的基准结果。 ### Mixed Precision (FP16) Training -细节请参考 [Mixed Precision (FP16) Training 在 PointPillars 训练的样例](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/pointpillars/hv_pointpillars_fpn_sbn-all_fp16_2x8_2x_nus-3d.py)。 +细节请参考 [Mixed Precision (FP16) Training 在 PointPillars 训练的样例](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/pointpillars/pointpillars_hv_fpn_sbn-all_8xb2-amp-2x_nus-3d.py)。 diff --git a/docs/zh_cn/notes/benchmarks.md b/docs/zh_cn/notes/benchmarks.md index b3b5c53ebb..52b9227076 100644 --- a/docs/zh_cn/notes/benchmarks.md +++ b/docs/zh_cn/notes/benchmarks.md @@ -25,7 +25,7 @@ ### 为了计算速度所做的修改 -- __MMDetection3D__：我们尝试使用与其他代码库中尽可能相同的配置，具体配置细节见 [基准测试配置](https://github.com/open-mmlab/MMDetection3D/blob/master/configs/benchmark)。 +- __MMDetection3D__：我们尝试使用与其他代码库中尽可能相同的配置，具体配置细节见 [基准测试配置](https://github.com/open-mmlab/MMDetection3D/blob/main/configs/benchmark)。 - __Det3D__：为了与 Det3D 进行比较，我们使用了 commit [519251e](https://github.com/poodarchu/Det3D/tree/519251e72a5c1fdd58972eabeac67808676b9bb7) 所对应的代码版本。 diff --git a/docs/zh_cn/user_guides/inference.md b/docs/zh_cn/user_guides/inference.md index 10bec99394..6e4cd30953 100644 --- a/docs/zh_cn/user_guides/inference.md +++ b/docs/zh_cn/user_guides/inference.md @@ -8,7 +8,7 @@ ### 3D 检测 -#### 单模态样例 +#### 点云样例 在点云数据上测试 3D 检测器，运行： @@ -18,59 +18,65 @@ python demo/pcd_demo.py ${PCD_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device 点云和预测 3D 框的可视化结果会被保存在 `${OUT_DIR}/PCD_NAME`，它可以使用 [MeshLab](http://www.meshlab.net/) 打开。注意如果你设置了 `--show`，通过 [Open3D](http://www.open3d.org/) 可以在线显示预测结果。 -在 KITTI 数据上测试 [SECOND](https://github.com/open-mmlab/mmdetection3d/tree/master/configs/second) 模型： +在 KITTI 数据上测试 [PointPillars 模型](https://download.openmmlab.com/mmdetection3d/v1.0.0_models/pointpillars/hv_pointpillars_secfpn_6x8_160e_kitti-3d-car/hv_pointpillars_secfpn_6x8_160e_kitti-3d-car_20220331_134606-d42d15ed.pth)： ```shell -python demo/pcd_demo.py demo/data/kitti/000008.bin configs/second/second_hv-secfpn_8xb6-80e_kitti-3d-car.py checkpoints/second_hv-secfpn_8xb6-80e_kitti-3d-car_20200620_230238-393f000c.pth +python demo/pcd_demo.py demo/data/kitti/000008.bin configs/pointpillars/pointpillars_hv_secfpn_8xb6-160e_kitti-3d-car.py ${CHECKPOINT_FILE} --show ``` -在 SUN RGB-D 数据上测试 [VoteNet](https://github.com/open-mmlab/mmdetection3d/tree/master/configs/votenet) 模型： +在 SUN RGB-D 数据上测试 [VoteNet 模型](https://download.openmmlab.com/mmdetection3d/v1.0.0_models/votenet/votenet_16x8_sunrgbd-3d-10class/votenet_16x8_sunrgbd-3d-10class_20210820_162823-bf11f014.pth)： ```shell -python demo/pcd_demo.py demo/data/sunrgbd/sunrgbd_000017.bin configs/votenet/votenet_8xb16_sunrgbd-3d.py checkpoints/votenet_8xb16_sunrgbd-3d_20200620_230238-4483c0c0.pth +python demo/pcd_demo.py demo/data/sunrgbd/sunrgbd_000017.bin configs/votenet/votenet_8xb16_sunrgbd-3d.py ${CHECKPOINT_FILE} --show ``` -如果你正在使用的 mmdetection3d 版本 >= 0.6.0，记住转换 VoteNet 的模型权重文件，查看 [README](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/votenet/README.md/) 来获取转换模型权重文件的详细说明。 +#### 单目 3D 样例 -#### 多模态样例 - -在多模态数据（通常是点云和图像）上测试 3D 检测器，运行： +在图像数据上测试单目 3D 检测器，运行： ```shell -python demo/multi_modality_demo.py ${PCD_FILE} ${IMAGE_FILE} ${ANNOTATION_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device ${GPU_ID}] [--score-thr ${SCORE_THR}] [--out-dir ${OUT_DIR}] [--show] +python demo/mono_det_demo.py ${IMAGE_FILE} ${ANNOTATION_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device ${GPU_ID}] [--out-dir ${OUT_DIR}] [--show] ``` -`ANNOTATION_FILE` 需要提供 3D 到 2D 的仿射矩阵，可视化结果会被保存在 `${OUT_DIR}/PCD_NAME`，其中包括点云、图像、预测的 3D 框以及它们在图像上的投影。 +`ANNOTATION_FILE` 需要提供 3D 到 2D 的仿射矩阵（相机内参矩阵），可视化结果会被保存在 `${OUT_DIR}/PCD_NAME`，其中包括图像以及预测 3D 框在图像上的投影。 -在 KITTI 数据上测试 [MVX-Net](https://github.com/open-mmlab/mmdetection3d/tree/master/configs/mvxnet) 模型： +在 KITTI 数据上测试 [PGD 模型](https://download.openmmlab.com/mmdetection3d/v1.0.0_models/pgd/pgd_r101_caffe_fpn_gn-head_3x4_4x_kitti-mono3d/pgd_r101_caffe_fpn_gn-head_3x4_4x_kitti-mono3d_20211022_102608-8a97533b.pth)： ```shell -python demo/multi_modality_demo.py demo/data/kitti/000008.bin demo/data/kitti/000008.png demo/data/kitti/000008.pkl configs/mvxnet/mvx_fpn-dv-second-secfpn_8xb2-80e_kitti-3d-3class.py checkpoints/mvx_fpn-dv-second-secfpn_8xb2-80e_kitti-3d-3class_20200621_003904-10140f2d.pth +python demo/mono_det_demo.py demo/data/kitti/000008.png demo/data/kitti/000008.pkl configs/pgd/pgd_r101-caffe_fpn_head-gn_4xb3-4x_kitti-mono3d.py ${CHECKPOINT_FILE} --show --cam-type CAM2 --score-thr 8 ``` -在 SUN RGB-D 数据上测试 [ImVoteNet](https://github.com/open-mmlab/mmdetection3d/tree/master/configs/imvotenet) 模型： +**注意**： PGD 方法的预测框分数并不是在 (0, 1) 之间 + +在 nuScenes 数据上测试 [FCOS3D 模型](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/fcos3d/fcos3d_r101_caffe_fpn_gn-head_dcn_2x8_1x_nus-mono3d_finetune/fcos3d_r101_caffe_fpn_gn-head_dcn_2x8_1x_nus-mono3d_finetune_20210717_095645-8d806dc2.pth)： ```shell -python demo/multi_modality_demo.py demo/data/sunrgbd/sunrgbd_000017.bin demo/data/sunrgbd/sunrgbd_000017.jpg demo/data/sunrgbd/sunrgbd_000017_infos.pkl configs/imvotenet/imvotenet_stage2_8xb16_sunrgbd.py checkpoints/imvotenet_stage2_8xb16_sunrgbd_20210323_184021-d44dcb66.pth +python demo/mono_det_demo.py demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__CAM_BACK__1532402927637525.jpg demo/data/nuscenes/n015-2018-07-24-11-22-45+0800.pkl configs/fcos3d/fcos3d_r101-caffe-dcn_fpn_head-gn_8xb2-1x_nus-mono3d_finetune.py ${CHECKPOINT_FILE} --show --cam-type CAM_BACK ``` -### 单目 3D 检测 +**注意**： 当对翻转图像可视化单目 3D 检测结果是，相机内参矩阵也应该相应修改。在 PR [#744](https://github.com/open-mmlab/mmdetection3d/pull/744) 中可以了解更多细节和示例。 -在图像数据上测试单目 3D 检测器，运行： +#### 多模态样例 + +在多模态数据（通常是点云和图像）上测试 3D 检测器，运行： ```shell -python demo/mono_det_demo.py ${IMAGE_FILE} ${ANNOTATION_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device ${GPU_ID}] [--out-dir ${OUT_DIR}] [--show] +python demo/multi_modality_demo.py ${PCD_FILE} ${IMAGE_FILE} ${ANNOTATION_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device ${GPU_ID}] [--score-thr ${SCORE_THR}] [--out-dir ${OUT_DIR}] [--show] ``` -`ANNOTATION_FILE` 需要提供 3D 到 2D 的仿射矩阵（相机内参矩阵），可视化结果会被保存在 `${OUT_DIR}/PCD_NAME`，其中包括图像以及预测 3D 框在图像上的投影。 +`ANNOTATION_FILE` 需要提供 3D 到 2D 的仿射矩阵，可视化结果会被保存在 `${OUT_DIR}/PCD_NAME`，其中包括点云、图像、预测的 3D 框以及它们在图像上的投影。 -在 nuScenes 数据上测试 [FCOS3D](https://github.com/open-mmlab/mmdetection3d/tree/master/configs/fcos3d) 模型： +在 KITTI 数据上测试 [MVX-Net](https://github.com/open-mmlab/mmdetection3d/tree/main/configs/mvxnet) 模型： ```shell -python demo/mono_det_demo.py demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__CAM_BACK__1532402927637525.jpg demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__CAM_BACK__1532402927637525.pkl configs/fcos3d/fcos3d_r101-caffe-dcn-fpn-head-gn_8xb2-1x_nus-mono3d_finetune.py checkpoints/fcos3d_r101-caffe-dcn-fpn-head-gn_8xb2-1x_nus-mono3d_finetune_20210717_095645-8d806dc2.pth +python demo/multi_modality_demo.py demo/data/kitti/000008.bin demo/data/kitti/000008.png demo/data/kitti/000008.pkl configs/mvxnet/mvxnet_fpn_dv_second_secfpn_8xb2-80e_kitti-3d-3class.py ${CHECKPOINT_FILE} --cam-type CAM2 --show ``` -注意当对翻转图像可视化单目 3D 检测结果是，相机内参矩阵也应该相应修改。在 PR [#744](https://github.com/open-mmlab/mmdetection3d/pull/744) 中可以了解更多细节和示例。 +在 SUN RGB-D 数据上测试 [ImVoteNet 模型](https://download.openmmlab.com/mmdetection3d/v1.0.0_models/imvotenet/imvotenet_stage2_16x8_sunrgbd-3d-10class/imvotenet_stage2_16x8_sunrgbd-3d-10class_20210819_192851-1bcd1b97.pth)： + +```shell +python demo/multi_modality_demo.py demo/data/sunrgbd/000017.bin demo/data/sunrgbd/000017.jpg demo/data/sunrgbd/sunrgbd_000017_infos.pkl configs/imvotenet/imvotenet_stage2_8xb16_sunrgbd-3d.py ${CHECKPOINT_FILE} --cam-type CAM0 --show --score-thr 0.6 +``` ### 3D 分割 @@ -82,8 +88,8 @@ python demo/pc_seg_demo.py ${PCD_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--devi 可视化结果会被保存在 `${OUT_DIR}/PCD_NAME`，其中包括点云以及预测的 3D 分割掩码。 -在 ScanNet 数据上测试 [PointNet++ (SSG)](https://github.com/open-mmlab/mmdetection3d/tree/master/configs/pointnet2) 模型： +在 ScanNet 数据上测试 [PointNet++ (SSG) 模型](https://download.openmmlab.com/mmdetection3d/v0.1.0_models/pointnet2/pointnet2_ssg_16x2_cosine_200e_scannet_seg-3d-20class/pointnet2_ssg_16x2_cosine_200e_scannet_seg-3d-20class_20210514_143644-ee73704a.pth)： ```shell -python demo/pc_seg_demo.py demo/data/scannet/scene0000_00.bin configs/pointnet2/pointnet2_ssg_2xb16-cosine-200e_scannet-seg.py checkpoints/pointnet2_ssg_2xb16-cosine-200e_scannet-seg_20210514_143644-ee73704a.pth +python demo/pcd_seg_demo.py demo/data/scannet/scene0000_00.bin configs/pointnet2/pointnet2_ssg_2xb16-cosine-200e_scannet-seg.py ${CHECKPOINT_FILE} --show ``` diff --git a/docs/zh_cn/user_guides/useful_tools.md b/docs/zh_cn/user_guides/useful_tools.md index 35554deff1..5cd256c73e 100644 --- a/docs/zh_cn/user_guides/useful_tools.md +++ b/docs/zh_cn/user_guides/useful_tools.md @@ -196,7 +196,7 @@ python -u tools/dataset_converters/nuimage_converter.py --data-root ${DATA_ROOT} - `--nproc`: 数据准备的进程数，默认为 `4`。由于图片是并行处理的，更大的进程数目能够减少准备时间。 - `--extra-tag`: 注释的额外标签，默认为 `nuimages`。这可用于将不同时间处理的不同注释分开以供研究。 -更多的数据准备细节参考 [doc](https://mmdetection3d.readthedocs.io/zh_CN/latest/data_preparation.html)，nuImages 数据集的细节参考 [README](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/nuimages/README.md/)。 +更多的数据准备细节参考 [doc](https://mmdetection3d.readthedocs.io/zh_CN/latest/data_preparation.html)，nuImages 数据集的细节参考 [README](https://github.com/open-mmlab/mmdetection3d/blob/main/configs/nuimages/README.md/)。   diff --git a/docs/zh_cn/user_guides/visualization.md b/docs/zh_cn/user_guides/visualization.md index 40de7f07da..6623d7ed28 100644 --- a/docs/zh_cn/user_guides/visualization.md +++ b/docs/zh_cn/user_guides/visualization.md @@ -190,7 +190,7 @@ python tools/misc/browse_dataset.py configs/mvxnet/dv_mvx-fpn_second_secfpn_adam 你可以使用不同的配置浏览不同的数据集，例如在 3D 语义分割任务中可视化 ScanNet 数据集： ```shell -python tools/misc/browse_dataset.py configs/_base_/datasets/scannet_seg-3d-20class.py --task lidar_seg --output-dir ${OUTPUT_DIR} --online +python tools/misc/browse_dataset.py configs/_base_/datasets/scannet-seg.py --task lidar_seg --output-dir ${OUTPUT_DIR} --online ``` ![](../../../resources/browse_dataset_seg.png) diff --git a/mmdet3d/apis/inference.py b/mmdet3d/apis/inference.py index 98b4373d07..2ba6eed23c 100644 --- a/mmdet3d/apis/inference.py +++ b/mmdet3d/apis/inference.py @@ -178,8 +178,8 @@ def inference_multi_modality_detector(model: nn.Module, ann_file: Union[str, Sequence[str]], cam_type: str = 'CAM2'): """Inference point cloud with the multi-modality detector. Now we only - support multi-modality detector for KITTI dataset since the multi-view - image loading is not supported yet in this inference function. + support multi-modality detector for KITTI and SUNRGBD datasets since the + multi-view image loading is not supported yet in this inference function. Args: model (nn.Module): The loaded detector. @@ -198,8 +198,6 @@ def inference_multi_modality_detector(model: nn.Module, If pcds is a list or tuple, the same length list type results will be returned, otherwise return the detection results directly. """ - - # TODO: We will support if isinstance(pcds, (list, tuple)): is_batch = True assert isinstance(imgs, (list, tuple)) @@ -229,9 +227,6 @@ def inference_multi_modality_detector(model: nn.Module, if osp.basename(img_path) != osp.basename(img): raise ValueError(f'the info file of {img_path} is not provided.') - data_info['images'][cam_type]['img_path'] = img - cam2img = np.array(data_info['images'][cam_type]['cam2img']) - # TODO: check the name consistency of # image file and point cloud file # TODO: support multi-view image loading @@ -239,8 +234,14 @@ def inference_multi_modality_detector(model: nn.Module, lidar_points=dict(lidar_path=pcd), img_path=img, box_type_3d=box_type_3d, - box_mode_3d=box_mode_3d, - cam2img=cam2img) + box_mode_3d=box_mode_3d) + + data_info['images'][cam_type]['img_path'] = img + if 'cam2img' in data_info['images'][cam_type]: + # The data annotation in SRUNRGBD dataset does not contain + # `cam2img` + data_['cam2img'] = np.array( + data_info['images'][cam_type]['cam2img']) # LiDAR to image conversion for KITTI dataset if box_mode_3d == Box3DMode.LIDAR: @@ -314,8 +315,10 @@ def inference_mono_3d_detector(model: nn.Module, # replace the img_path in data_info with img data_info['images'][cam_type]['img_path'] = img + # avoid data_info['images'] has multiple keys anout camera views. + mono_img_info = {f'{cam_type}': data_info['images'][cam_type]} data_ = dict( - images=data_info['images'], + images=mono_img_info, box_type_3d=box_type_3d, box_mode_3d=box_mode_3d) diff --git a/mmdet3d/datasets/transforms/formating.py b/mmdet3d/datasets/transforms/formating.py index 477346a59b..c1b1dece4b 100644 --- a/mmdet3d/datasets/transforms/formating.py +++ b/mmdet3d/datasets/transforms/formating.py @@ -195,11 +195,29 @@ def pack_single_results(self, results: dict) -> dict: gt_instances = InstanceData() gt_pts_seg = PointData() - img_metas = {} + data_metas = {} for key in self.meta_keys: if key in results: - img_metas[key] = results[key] - data_sample.set_metainfo(img_metas) + data_metas[key] = results[key] + elif 'images' in results: + if len(results['images'].keys()) == 1: + cam_type = list(results['images'].keys())[0] + # single-view image + if key in results['images'][cam_type]: + data_metas[key] = results['images'][cam_type][key] + else: + # multi-view image + img_metas = [] + cam_types = list(results['images'].keys()) + for cam_type in cam_types: + if key in results['images'][cam_type]: + img_metas.append(results['images'][cam_type][key]) + if len(img_metas) > 0: + data_metas[key] = img_metas + elif 'lidar_points' in results: + if key in results['lidar_points']: + data_metas[key] = results['lidar_points'][key] + data_sample.set_metainfo(data_metas) inputs = {} for key in self.keys: diff --git a/mmdet3d/datasets/transforms/loading.py b/mmdet3d/datasets/transforms/loading.py index a240a1c032..025c429124 100644 --- a/mmdet3d/datasets/transforms/loading.py +++ b/mmdet3d/datasets/transforms/loading.py @@ -250,7 +250,7 @@ def transform(self, results: dict) -> dict: results['cam2img'] = results['images'][camera_type]['cam2img'] else: raise NotImplementedError( - 'Currently we only support load image from kitti and' + 'Currently we only support load image from kitti and ' 'nuscenes datasets') try: diff --git a/mmdet3d/engine/hooks/visualization_hook.py b/mmdet3d/engine/hooks/visualization_hook.py index 34c81ef5cf..5fc46ba6e6 100644 --- a/mmdet3d/engine/hooks/visualization_hook.py +++ b/mmdet3d/engine/hooks/visualization_hook.py @@ -97,13 +97,18 @@ def after_val_iter(self, runner: Runner, batch_idx: int, data_batch: dict, data_input = dict() # Visualize only the first data - if 'img_path' in outputs[0]: + if self.vis_task in [ + 'mono_det', 'multi-view_det', 'multi-modality_det' + ]: + assert 'img_path' in outputs[0], 'img_path is not in outputs[0]' img_path = outputs[0].img_path img_bytes = get(img_path, backend_args=self.backend_args) img = mmcv.imfrombytes(img_bytes, channel_order='rgb') data_input['img'] = img - if 'lidar_path' in outputs[0]: + if self.vis_task in ['lidar_det', 'multi-modality_det', 'lidar_seg']: + assert 'lidar_path' in outputs[ + 0], 'lidar_path is not in outputs[0]' lidar_path = outputs[0].lidar_path num_pts_feats = outputs[0].num_pts_feats pts_bytes = get(lidar_path, backend_args=self.backend_args) @@ -145,24 +150,39 @@ def after_test_iter(self, runner: Runner, batch_idx: int, data_batch: dict, self._test_index += 1 data_input = dict() - if 'img_path' in data_sample: + assert 'img_path' in data_sample or 'lidar_path' in data_sample, \ + "'data_sample' must contain 'img_path' or 'lidar_path'" + + out_file = o3d_save_path = None + + if self.vis_task in [ + 'mono_det', 'multi-view_det', 'multi-modality_det' + ]: + assert 'img_path' in data_sample, \ + 'img_path is not in data_sample' img_path = data_sample.img_path img_bytes = get(img_path, backend_args=self.backend_args) img = mmcv.imfrombytes(img_bytes, channel_order='rgb') data_input['img'] = img - - if 'lidar_path' in data_sample: + if self.test_out_dir is not None: + out_file = osp.basename(img_path) + out_file = osp.join(self.test_out_dir, out_file) + + if self.vis_task in [ + 'lidar_det', 'multi-modality_det', 'lidar_seg' + ]: + assert 'lidar_path' in data_sample, \ + 'lidar_path is not in data_sample' lidar_path = data_sample.lidar_path num_pts_feats = data_sample.num_pts_feats pts_bytes = get(lidar_path, backend_args=self.backend_args) points = np.frombuffer(pts_bytes, dtype=np.float32) points = points.reshape(-1, num_pts_feats) data_input['points'] = points - - out_file = None - if self.test_out_dir is not None: - out_file = osp.basename(img_path) - out_file = osp.join(self.test_out_dir, out_file) + if self.test_out_dir is not None: + o3d_save_path = osp.basename(lidar_path).split( + '.')[0] + '.png' + o3d_save_path = osp.join(self.test_out_dir, o3d_save_path) self._visualizer.add_datasample( 'test sample', @@ -173,4 +193,5 @@ def after_test_iter(self, runner: Runner, batch_idx: int, data_batch: dict, wait_time=self.wait_time, pred_score_thr=self.score_thr, out_file=out_file, + o3d_save_path=o3d_save_path, step=self._test_index) diff --git a/mmdet3d/visualization/local_visualizer.py b/mmdet3d/visualization/local_visualizer.py index ec53e6ad74..472bc98ecd 100644 --- a/mmdet3d/visualization/local_visualizer.py +++ b/mmdet3d/visualization/local_visualizer.py @@ -664,6 +664,9 @@ def show(self, if hasattr(self, 'o3d_vis'): self.o3d_vis.run() if save_path is not None: + if not (save_path.endswith('.png') + or save_path.endswith('.jpg')): + save_path += '.png' self.o3d_vis.capture_screen_image(save_path) self.o3d_vis.destroy_window() self._clear_o3d_vis() diff --git a/tools/test.py b/tools/test.py index d49477cfd8..794570224b 100644 --- a/tools/test.py +++ b/tools/test.py @@ -28,6 +28,8 @@ def parse_args(): help='directory where painted images will be saved. ' 'If specified, it will be automatically saved ' 'to the work_dir/timestamp/show_dir') + parser.add_argument( + '--score-thr', type=float, default=0.1, help='bbox score threshold') parser.add_argument( '--task', type=str, @@ -71,7 +73,15 @@ def trigger_visualization_hook(cfg, args): visualization_hook['wait_time'] = args.wait_time if args.show_dir: visualization_hook['test_out_dir'] = args.show_dir + all_task_choices = [ + 'mono_det', 'multi-view_det', 'lidar_det', 'lidar_seg', + 'multi-modality_det' + ] + assert args.task in all_task_choices, 'You must set '\ + f"'--task' in {all_task_choices} in the command " \ + 'if you want to use visualization hook' visualization_hook['vis_task'] = args.task + visualization_hook['score_thr'] = args.score_thr else: raise RuntimeError( 'VisualizationHook must be included in default_hooks.'