Feature/first draft #2
Conversation
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
|
Finish fixing first feedback |
|
|
||
| ## Sensor fusion with `RadarScan` in detection module | ||
|
|
||
| In addition to Camera-LiDAR sensor fusion in the detection layer proposed in [Perception architecture discussion](https://github.com/autowarefoundation/autoware/discussions/3), I suggest including Camera-LiDAR-Radar sensor fusion in the detection layer. |
There was a problem hiding this comment.
In addition to Camera-LiDAR sensor fusion in the detection layer proposed in Perception architecture discussion, this document proposes including radar in the detection layer (camera-lidar-radar).
|
|
||
| ### Camera-LiDAR-Radar 3d detection using multiple modules | ||
|
|
||
| Camera-LiDAR-Radar 3D detection consists of two sensor fusion stages. |
There was a problem hiding this comment.
The proposed Camera-LiDAR-Radar 3D detection pipeline consist of two sensor fusion stages:
|
|
||
| The first stage is Camera-LiDAR fusion. | ||
| Camera-LiDAR fusion packages aim to improve class label accuracy. | ||
| Detected 2D objects and detected 3D objects are fused in Camera-LiDAR Fusion methods like [roi_cluster_fusion](https://github.com/autowarefoundation/autoware.universe/tree/main/perception/roi_cluster_fusion) and [image_projection_based_fusion in the future](https://github.com/autowarefoundation/autoware.universe/pull/548). |
There was a problem hiding this comment.
Independently detected 2D and 3D objects are fused in the Camera-LiDAR Fusion module using methods such as:
- roi_cluster_fusion. ADD A BRIEF DESCRIPTION OF THIS NODE.
- image_projection_based_fusion in the future. ADD A BRIEF DESCRIPTION OF THIS NODE.
There was a problem hiding this comment.
I merge your review and update of autoware.universe (PR is merged)
| - Radar fusion | ||
|
|
||
| The second stage is radar fusion. | ||
| After Camera-LiDAR fusion, detected 3D objects from Camera-LiDAR fusion and `RadarScan` are fused in radar fusion modules. |
There was a problem hiding this comment.
and
RadarScanare fused in radar fusion modules.
How? Please add a link to the document or explanation.
There was a problem hiding this comment.
For now, detail design is not decided yet.
I think to add link to a package after making prototype.
| Radar fusion packages aim to improve detection performance using detected 3D objects with low confidence and `RadarScan`. | ||
| Radar fusion packages attach doppler velocity from `RadarScan` for detected 3D objects to improve velocity estimation in the tracking module. | ||
|
|
||
| By defining `DetectedObjects` as the input interface for camera and LiDAR data, it can improve usability for sensor fusion architecture; for example, the radar fusion module can be applied to the camera 3D detection. |
There was a problem hiding this comment.
it can improve usability for sensor fusion architecture;
Can you please explain further? why?
There was a problem hiding this comment.
There are many combinations if message types are prepared for each sensors.
For example,
- Camera image (sensor_msgs/Image) - Radar pointcloud:
- LiDAR pointcluod (sensor_msgs/Pointcloud2) - Radar pointcloud
- Camera 2D object (2D ClassfiledRoi) - Radar pointcloud:
- Camera 3D object (DetectedObjects) - Radar pointcloud:
- LiDAR 3D object (DetectedObjects) - Radar pointcloud
It leads to complicated architecture.
So, for the first step, I recommend sensor fusion with consistent messages type, DetectedObjects.
| As described in the radar sensing proposal, radar output can be divided into two types; scan and objects. | ||
| According to these output types, I suggest two sensor fusion types for radar output: | ||
|
|
||
| - Radar fusion in the object layer in the tracking module |
There was a problem hiding this comment.
Can you please add further explanations on these two points?
There was a problem hiding this comment.
This two points are just summary.
Further explanations are put in radar perception design
I think redundant explanations in summary leads to make overall outlook worse.
proposal_radar_sensing.md
Outdated
| @@ -0,0 +1,55 @@ | |||
| ## Summary | |||
|
|
|||
| I would like to initiate the discussion for sensing architecture with radars for Autoware. | |||
There was a problem hiding this comment.
I find it weird you use the first singular personal pronoun on a proposal.
Is there any reason you want to use "I"?
|
|
||
| To summarize, I suggest | ||
|
|
||
| - [ros-perception/radar_msgs/msg/RadarScan.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarScan.msg) for radar pointcloud. |
There was a problem hiding this comment.
Similar to the perception proposal, can you please add why these?
There was a problem hiding this comment.
I just wrote the reasons in radar pointcloud message design and radar object message design.
|
|
||
|  | ||
|
|
||
| This figure can be found at <https://github.com/scepter914/autoware-radar-architecture-proposal/blob/main/sensing/figure/sensing_pipeline.drawio.svg>. |
There was a problem hiding this comment.
I think adding a description of your figures is essential to guide the reader.
There was a problem hiding this comment.
I intend to follow to the Sensing and perception architecture proposal.
And I think this document is not in charge for detail documents to whole pipeline.
(I also think design documents in sensing and perception is needed.)
|
|
||
| ### Whole pipeline | ||
|
|
||
| The proposed pipeline follows the [Sensing and perception architecture proposal](https://github.com/autowarefoundation/autoware/discussions/3). |
There was a problem hiding this comment.
- How does your proposal follow AWF?
- Why are you following it?
- Which aspects are you bringing to yours?
There was a problem hiding this comment.
AWF discuss now proposed architecture in this link (The link of this slide is written in Sensing and perception architecture proposal, so I think information for whole pipeline is enough.)
And I add radar pipeline to it.
proposal_radar_sensing.md
Outdated
|
|
||
| ### Radar driver | ||
|
|
||
| For now, I suggest that Autoware radar drivers should support `ros-perception/radar_msgs/msg/RadarScan.msg` and `autoware_auto_perception_msgs/msg/TrackedObjects`, because these two outputs are more useful for sensor fusion in the sensing and perception module than others. |
There was a problem hiding this comment.
because these two outputs are more useful
How do you know how are these more useful?
There was a problem hiding this comment.
Two outputs can be applied to many sensor fusion algorithm.
Many radar fusion algorithm use radar pointclouds and radar tracked objects in detection and tracking tasks.
|
|
||
| To sum up, I suggest: | ||
|
|
||
| - In the sensing layer, the radar preprocess package filters noise through the `ros-perception/radar_msgs/msg/RadarScan.msg` message type. |
There was a problem hiding this comment.
the radar preprocess
Can you please add a link to the description of this preprocessing?
The same for the converter RadarScan->Pointcloud
There was a problem hiding this comment.
If there are other projects already available or any papers you have in mind, it's ok to add them here.
There was a problem hiding this comment.
there are no detail design and I think detail design should not be written in this proposal, but written in design document after prototype tests.
| The radar sensing package processes `RadarScan` topics. | ||
| For `RadarTracks` topics, please see [perception architecture design](https://github.com/scepter914/autoware-radar-architecture-proposal/blob/main/perception/radar_perception_design.md). | ||
|
|
||
|  |
There was a problem hiding this comment.
Similarly, please add a description of this Figure.
There was a problem hiding this comment.
I explained in Node section.
| As a first prototype of radar preprocess, I suggest | ||
|
|
||
| - Radar threshold filter | ||
| - This package removes noise (low intensity and edge angle of sensors) from radar points. |
There was a problem hiding this comment.
How? Do you have any references?
There was a problem hiding this comment.
There are no reference.
I'll try noise filter in prototype test.
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
Signed-off-by: scepter914 <scepter914@gmail.com>
|
I update from Abraham's review. |
Signed-off-by: scepter914 <scepter914@gmail.com>
|
As a result of consultation with David, I send the proposals for AWF. But we can update detail documents in this repository, so I would like to continue to discuss the design and documents. |
Signed-off-by: scepter914 <scepter914@gmail.com>
|
Thank you very much for honest reviews. then I would like to continue discussing in AWF. |
For tier4 member feedback