[Docs] Update readme of lidar segmentation methods (#2559)

* add readme * fix spvcnn memory * add fps and training time * resolve typo * fix typo
open-mmlab · May 30, 2023 · 56657c2 · 56657c2
1 parent fa724b1
commit 56657c2
Show file tree

Hide file tree

Showing 6 changed files with 152 additions and 29 deletions.
diff --git a/configs/cylinder3d/README.md b/configs/cylinder3d/README.md
@@ -18,9 +18,10 @@ We implement Cylinder3D and provide the result and checkpoints on Semantickitti
 
 ### SemanticKITTI
 
-|   Method   | Lr schd | Mem (GB) |   mIOU   |                                                                                                                                                                       Download                                                                                                                                                                       |
-| :--------: | :-----: | :------: | :------: | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
-| Cylinder3D |   3x    |   10.2   | 63.1±0.5 | [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/cylinder3d/cylinder3d_4xb4_3x_semantickitti/cylinder3d_4xb4_3x_semantickitti_20230318_191107-822a8c31.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/cylinder3d/cylinder3d_4xb4_3x_semantickitti/cylinder3d_4xb4_3x_semantickitti_20230318_191107.json) |
+|                               Method                                | Lr schd | Laser-Polar Mix | Mem (GB) |   mIoU   |                                                                                                                                                                       Download                                                                                                                                                                       |
+| :-----------------------------------------------------------------: | :-----: | :-------------: | :------: | :------: | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
+| [Cylinder3D](./cylinder3d_8xb2-laser-polar-mix-3x_semantickitti.py) |   3x    |        ✗        |   10.2   | 63.1±0.5 | [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/cylinder3d/cylinder3d_4xb4_3x_semantickitti/cylinder3d_4xb4_3x_semantickitti_20230318_191107-822a8c31.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/cylinder3d/cylinder3d_4xb4_3x_semantickitti/cylinder3d_4xb4_3x_semantickitti_20230318_191107.json) |
+| [Cylinder3D](./cylinder3d_8xb2-laser-polar-mix-3x_semantickitti.py) |   3x    |        ✔        |   12.8   |   67.0   |              [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/cylinder3d/cylinder3d_8xb2-amp-laser-polar-mix-3x_semantickitti_20230425_144950-372cdf69.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/cylinder3d/cylinder3d_8xb2-amp-laser-polar-mix-3x_semantickitti_20230425_144950.log)               |
 
 Note: We reproduce the performance comparable with its [official repo](https://github.com/xinge008/Cylinder3D). It's slightly lower than the performance (65.9 mIOU) reported in the paper due to the lack of point-wise refinement and shorter training time.
 

diff --git a/configs/cylinder3d/metafile.yml b/configs/cylinder3d/metafile.yml
@@ -15,7 +15,7 @@ Collections:
       Version: v1.1.0
 
 Models:
-  - Name:
+  - Name: cylinder3d_4xb4-3x_semantickitti
     In Collection: Cylinder3D
     Config: configs/cylinder3d/cylinder3d_4xb4_3x_semantickitti.py
     Metadata:
@@ -25,5 +25,18 @@ Models:
       - Task: 3D Semantic Segmentation
         Dataset: SemanticKITTI
         Metrics:
-          mIOU: 63.1
-    Weights:
+          mIoU: 63.1
+    Weights: https://download.openmmlab.com/mmdetection3d/v1.1.0_models/cylinder3d/cylinder3d_4xb4_3x_semantickitti/cylinder3d_4xb4_3x_semantickitti_20230318_191107-822a8c31.pth
+
+  - Name: cylinder3d_8xb2-laser-polar-mix-3x_semantickitti
+    In Collection: Cylinder3D
+    Config: configs/cylinder3d/cylinder3d_8xb2-laser-polar-mix-3x_semantickitti.py
+    Metadata:
+      Training Data: SemanticKITTI
+      Training Memory (GB): 12.8
+    Results:
+      - Task: 3D Semantic Segmentation
+        Dataset: SemanticKITTI
+        Metrics:
+          mIoU: 67.0
+    Weights: https://download.openmmlab.com/mmdetection3d/v1.1.0_models/cylinder3d/cylinder3d_4xb4_3x_semantickitti/cylinder3d_4xb4_3x_semantickitti_20230318_191107-822a8c31.pth
diff --git a/configs/minkunet/README.md b/configs/minkunet/README.md
@@ -14,22 +14,32 @@ In many robotics and VR/AR applications, 3D-videos are readily-available sources
 
 ## Introduction
 
-We implement MinkUNet with [TorchSparse](https://github.com/mit-han-lab/torchsparse) backend and provide the result and checkpoints on SemanticKITTI datasets.
+We implement MinkUNet with [TorchSparse](https://github.com/mit-han-lab/torchsparse) / [Minkowski Engine](https://github.com/NVIDIA/MinkowskiEngine) / [Spconv](https://github.com/traveller59/spconv) backend and provide the result and checkpoints on SemanticKITTI datasets.
 
 ## Results and models
 
 ### SemanticKITTI
 
-|    Method    | Lr schd | Mem (GB) | mIoU |                                                                                                                                                                          Download                                                                                                                                                                           |
-| :----------: | :-----: | :------: | :--: | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
-| MinkUNet-W16 |   15e   |   3.4    | 60.3 | [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w16_8xb2-15e_semantickitti/minkunet_w16_8xb2-15e_semantickitti_20230309_160737-0d8ec25b.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w16_8xb2-15e_semantickitti/minkunet_w16_8xb2-15e_semantickitti_20230309_160737.log) |
-| MinkUNet-W20 |   15e   |   3.7    | 61.6 | [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w20_8xb2-15e_semantickitti/minkunet_w20_8xb2-15e_semantickitti_20230309_160718-c3b92e6e.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w20_8xb2-15e_semantickitti/minkunet_w20_8xb2-15e_semantickitti_20230309_160718.log) |
-| MinkUNet-W32 |   15e   |   4.9    | 63.1 | [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w32_8xb2-15e_semantickitti/minkunet_w32_8xb2-15e_semantickitti_20230309_160710-7fa0a6f1.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w32_8xb2-15e_semantickitti/minkunet_w32_8xb2-15e_semantickitti_20230309_160710.log) |
+|                                            Method                                             |     Backend      | Lr schd | Amp | Laser-Polar Mix | Mem (GB) | Training Time (hours) |  FPS   | mIoU |                                                                                                                                                                          Download                                                                                                                                                                           |
+| :-------------------------------------------------------------------------------------------: | :--------------: | :-----: | :-: | :-------------: | :------: | :-------------------: | :----: | :--: | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
+|         [MinkUNet18-W16](./minkunet18_w16_torchsparse_8xb2-amp-15e_semantickitti.py)          |   torchsparse    |   15e   |  ✔  |        ✗        |   3.4    |           -           |   -    | 60.3 | [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w16_8xb2-15e_semantickitti/minkunet_w16_8xb2-15e_semantickitti_20230309_160737-0d8ec25b.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w16_8xb2-15e_semantickitti/minkunet_w16_8xb2-15e_semantickitti_20230309_160737.log) |
+|         [MinkUNet18-W20](./minkunet18_w20_torchsparse_8xb2-amp-15e_semantickitti.py)          |   torchsparse    |   15e   |  ✔  |        ✗        |   3.7    |           -           |   -    | 61.6 | [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w20_8xb2-15e_semantickitti/minkunet_w20_8xb2-15e_semantickitti_20230309_160718-c3b92e6e.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w20_8xb2-15e_semantickitti/minkunet_w20_8xb2-15e_semantickitti_20230309_160718.log) |
+|         [MinkUNet18-W32](./minkunet18_w32_torchsparse_8xb2-amp-15e_semantickitti.py)          |   torchsparse    |   15e   |  ✔  |        ✗        |   4.9    |           -           |   -    | 63.1 | [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w32_8xb2-15e_semantickitti/minkunet_w32_8xb2-15e_semantickitti_20230309_160710-7fa0a6f1.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w32_8xb2-15e_semantickitti/minkunet_w32_8xb2-15e_semantickitti_20230309_160710.log) |
+|     [MinkUNet34-W32](./minkunet34_w32_minkowski_8xb2-laser-polar-mix-3x_semantickitti.py)     | minkowski engine |   3x    |  ✗  |        ✔        |   11.5   |          6.5          |  12.2  | 69.2 |          [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34_w32_minkowski_8xb2-laser-polar-mix-3x_semantickitti_20230514_202236-839847a8.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34_w32_minkowski_8xb2-laser-polar-mix-3x_semantickitti_20230514_202236.log)          |
+|    [MinkUNet34-W32](./minkunet34_w32_spconv_8xb2-amp-laser-polar-mix-3x_semantickitti.py)     |      spconv      |   3x    |  ✔  |        ✔        |   6.7    |           2           | 14.6\* | 68.3 |         [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34_w32_spconv_8xb2-amp-laser-polar-mix-3x_semantickitti_20230512_233152-e0698a0f.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34_w32_spconv_8xb2-amp-laser-polar-mix-3x_semantickitti_20230512_233152.log)         |
+|      [MinkUNet34-W32](./minkunet34_w32_spconv_8xb2-laser-polar-mix-3x_semantickitti.py)       |      spconv      |   3x    |  ✗  |        ✔        |   10.5   |           6           |  14.5  |  3   |                                                                                                                                                                            69.3                                                                                                                                                                             |
+|  [MinkUNet34-W32](./minkunet34_w32_torchsparse_8xb2-amp-laser-polar-mix-3x_semantickitti.py)  |   torchsparse    |   3x    |  ✔  |        ✔        |   6.6    |           3           |  12.8  | 69.3 |    [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34_w32_torchsparse_8xb2-amp-laser-polar-mix-3x_semantickitti_20230512_233511-bef6cad0.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34_w32_torchsparse_8xb2-amp-laser-polar-mix-3x_semantickitti_20230512_233511.log)    |
+|    [MinkUNet34-W32](./minkunet34_w32_torchsparse_8xb2-laser-polar-mix-3x_semantickitti.py)    |   torchsparse    |   3x    |  ✗  |        ✔        |   11.8   |          5.5          |  15.9  | 68.7 |        [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34_w32_torchsparse_8xb2-laser-polar-mix-3x_semantickitti_20230512_233601-2b61b0ab.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34_w32_torchsparse_8xb2-laser-polar-mix-3x_semantickitti_20230512_233601.log)        |
+| [MinkUNet34v2-W32](minkunet34v2_w32_torchsparse_8xb2-amp-laser-polar-mix-3x_semantickitti.py) |   torchsparse    |   3x    |  ✔  |        ✔        |   8.9    |           -           |   -    | 70.3 |  [model](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34v2_w32_torchsparse_8xb2-amp-laser-polar-mix-3x_semantickitti_20230510_221853-b14a68b3.pth) \| [log](https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34v2_w32_torchsparse_8xb2-amp-laser-polar-mix-3x_semantickitti_20230510_221853.log)  |
 
 **Note:** We follow the implementation in SPVNAS original [repo](https://github.com/mit-han-lab/spvnas) and W16\\W20\\W32 indicates different number of channels.
 
 **Note:** Due to TorchSparse backend, the model performance is unstable with TorchSparse backend and may fluctuate by about 1.5 mIoU for different random seeds.
 
+**Note:** Referring to [PCSeg](https://github.com/PJLab-ADG/PCSeg), MinkUNet34v2 is modified based on MinkUNet34.
+
+**Note\*:** Training Time and FPS are measured on NVIDIA A100. The versions of Torchsparse, Minkowski Engine and Spconv are 0.5.4, 1.4.0 and 2.3.6 respectively. Since spconv 2.3.6 has a bug with fp16 on in the inference stage, the actual FPS measurement using fp32.
+
 ## Citation
 
 ```latex

diff --git a/configs/minkunet/metafile.yml b/configs/minkunet/metafile.yml
@@ -14,9 +14,9 @@ Collections:
       Version: v1.1.0
 
 Models:
-  - Name: minkunet_w16_8xb2-15e_semantickitti
+  - Name: minkunet18_w16_torchsparse_8xb2-amp-15e_semantickitti
     In Collection: MinkUNet
-    Config: configs/minkunet/minkunet_w16_8xb2-15e_semantickitti.py
+    Config: configs/minkunet/minkunet18_w16_torchsparse_8xb2-amp-15e_semantickitti.py
     Metadata:
       Training Data: SemanticKITTI
       Training Memory (GB): 3.4
@@ -28,9 +28,9 @@ Models:
           mIoU: 60.3
     Weights: https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w16_8xb2-15e_semantickitti/minkunet_w16_8xb2-15e_semantickitti_20230309_160737-0d8ec25b.pth
 
-  - Name: minkunet_w20_8xb2-15e_semantickitti
+  - Name: minkunet18_w20_torchsparse_8xb2-amp-15e_semantickitti
     In Collection: MinkUNet
-    Config: configs/minkunet/minkunet_w20_8xb2-15e_semantickitti.py
+    Config: configs/minkunet/minkunet18_w20_torchsparse_8xb2-amp-15e_semantickitti.py
     Metadata:
       Training Data: SemanticKITTI
       Training Memory (GB): 3.7
@@ -42,9 +42,9 @@ Models:
           mIoU: 61.6
     Weights: https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w20_8xb2-15e_semantickitti/minkunet_w20_8xb2-15e_semantickitti_20230309_160718-c3b92e6e.pth
 
-  - Name: minkunet_w32_8xb2-15e_semantickitti
+  - Name: minkunet18_w32_torchsparse_8xb2-amp-15e_semantickitti
     In Collection: MinkUNet
-    Config: configs/minkunet/minkunet_w32_8xb2-15e_semantickitti.py
+    Config: configs/minkunet/minkunet18_w32_torchsparse_8xb2-amp-15e_semantickitti.py
     Metadata:
       Training Data: SemanticKITTI
       Training Memory (GB): 4.9
@@ -55,3 +55,87 @@ Models:
         Metrics:
           mIoU: 63.1
     Weights: https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet_w32_8xb2-15e_semantickitti/minkunet_w32_8xb2-15e_semantickitti_20230309_160710-7fa0a6f1.pth
+
+  - Name: minkunet34_w32_minkowski_8xb2-laser-polar-mix-3x_semantickitti
+    In Collection: MinkUNet
+    Config: configs/minkunet/minkunet34_w32_minkowski_8xb2-laser-polar-mix-3x_semantickitti.py
+    Metadata:
+      Training Data: SemanticKITTI
+      Training Memory (GB): 11.5
+      Training Resources: 8x A100 GPUs
+    Results:
+      - Task: 3D Semantic Segmentation
+        Dataset: SemanticKITTI
+        Metrics:
+          mIoU: 69.2
+    Weights: https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34_w32_minkowski_8xb2-laser-polar-mix-3x_semantickitti_20230514_202236-839847a8.pth
+
+  - Name: minkunet34_w32_spconv_8xb2-amp-laser-polar-mix-3x_semantickitti
+    In Collection: MinkUNet
+    Config: configs/minkunet/minkunet34_w32_spconv_8xb2-amp-laser-polar-mix-3x_semantickitti.py
+    Metadata:
+      Training Data: SemanticKITTI
+      Training Memory (GB): 6.7
+      Training Resources: 8x A100 GPUs
+    Results:
+      - Task: 3D Semantic Segmentation
+        Dataset: SemanticKITTI
+        Metrics:
+          mIoU: 68.3
+    Weights: https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34_w32_spconv_8xb2-amp-laser-polar-mix-3x_semantickitti_20230512_233152-e0698a0f.pth
+
+  - Name: minkunet34_w32_spconv_8xb2-laser-polar-mix-3x_semantickitti
+    In Collection: MinkUNet
+    Config: configs/minkunet/minkunet34_w32_spconv_8xb2-laser-polar-mix-3x_semantickitti.py
+    Metadata:
+      Training Data: SemanticKITTI
+      Training Memory (GB): 10.5
+      Training Resources: 8x A100 GPUs
+    Results:
+      - Task: 3D Semantic Segmentation
+        Dataset: SemanticKITTI
+        Metrics:
+          mIoU: 69.3
+    Weights: https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34_w32_spconv_8xb2-laser-polar-mix-3x_semantickitti_20230512_233817-72b200d8.pth
+
+  - Name: minkunet34_w32_torchsparse_8xb2-amp-laser-polar-mix-3x_semantickitti
+    In Collection: MinkUNet
+    Config: configs/minkunet/minkunet34_w32_torchsparse_8xb2-amp-laser-polar-mix-3x_semantickitti.py
+    Metadata:
+      Training Data: SemanticKITTI
+      Training Memory (GB): 6.6
+      Training Resources: 8x A100 GPUs
+    Results:
+      - Task: 3D Semantic Segmentation
+        Dataset: SemanticKITTI
+        Metrics:
+          mIoU: 69.3
+    Weights: https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34_w32_torchsparse_8xb2-amp-laser-polar-mix-3x_semantickitti_20230512_233511-bef6cad0.pth
+
+  - Name: minkunet34_w32_torchsparse_8xb2-laser-polar-mix-3x_semantickitti
+    In Collection: MinkUNet
+    Config: configs/minkunet/minkunet34_w32_torchsparse_8xb2-laser-polar-mix-3x_semantickitti.py
+    Metadata:
+      Training Data: SemanticKITTI
+      Training Memory (GB): 11.8
+      Training Resources: 8x A100 GPUs
+    Results:
+      - Task: 3D Semantic Segmentation
+        Dataset: SemanticKITTI
+        Metrics:
+          mIoU: 68.7
+    Weights: https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34_w32_torchsparse_8xb2-laser-polar-mix-3x_semantickitti_20230512_233601-2b61b0ab.pth
+
+  - Name: minkunet34v2_w32_torchsparse_8xb2-amp-laser-polar-mix-3x_semantickitti
+    In Collection: MinkUNet
+    Config: configs/minkunet/minkunet34v2_w32_torchsparse_8xb2-amp-laser-polar-mix-3x_semantickitti.py
+    Metadata:
+      Training Data: SemanticKITTI
+      Training Memory (GB): 8.9
+      Training Resources: 8x A100 GPUs
+    Results:
+      - Task: 3D Semantic Segmentation
+        Dataset: SemanticKITTI
+        Metrics:
+          mIoU: 70.3
+    Weights: https://download.openmmlab.com/mmdetection3d/v1.1.0_models/minkunet/minkunet34v2_w32_torchsparse_8xb2-amp-laser-polar-mix-3x_semantickitti_20230510_221853-b14a68b3.pth