Skip to content

Latest commit

 

History

History
78 lines (58 loc) · 3.39 KB

README.md

File metadata and controls

78 lines (58 loc) · 3.39 KB
Raw

kitti_native_evaluation

Disclaimer:

  • These days, I found this code to be too slow for the turnover rate in deep learning development. I personally switched to using numba as done in https://github.com/traveller59/second.pytorch to compute AP in much faster times (~10 seconds per full run on validation set). I recommend you take a look if you need faster code with python integration.

Updates:

  • 17/12/2017: Initial release.
  • 18/02/2020: This code complies with the 40 recall point change on the official KITTI website.
  • 18/02/2020: I am not actively developing this repository anymore. This will be the last update for the time being.

This code is based on Bo Li's repository: https://github.com/prclibo/kitti_eval with the main differences being some code cleanup and additional AHS metric described in our paper: Joint 3D Proposal Generation and Object Detection from View Aggregation

evaluate_object_3d_offline.cpp evaluates your KITTI detection locally on your own computer using your validation data selected from KITTI training dataset, with the following metrics:

  • overlap on image (AP)
  • oriented overlap on image (AOS)
  • overlap on ground-plane (AP)
  • oriented overlap on ground-plane (AHS)
  • overlap in 3D (AP)
  • oriented overlap in 3D (3D AHS)

Compilation:

Clone the repo using: git clone https://github.com/zenseact/kitti_native_evaluation.git

From inside the main folder do:

  1. cmake ./
  2. make

Note from Zenseact team: We also had to run sudo apt-get install libboost-all-dev

Usage:

Run the evalutaion by:

./evaluate_object_3d_offline groundtruth_dir result_dir

Note that you don't have to detect over all KITTI training data. The evaluator only evaluates samples whose result files exist.

Data Format:

The detection format should be simillar to the KITTI dataset label format with 15 columns representing:

Values Name Description
1 type Describes the type of object: 'Car', 'Van', 'Truck', 'Pedestrian', 'Person_sitting', 'Cyclist', 'Tram', 'Misc' or 'DontCare'
1 truncated -1
1 occluded -1
1 alpha Observation angle of object, ranging [-pi..pi]
4 bbox 2D bounding box of object in the image (0-based index): contains left, top, right, bottom pixel coordinates
3 dimensions 3D object dimensions: height, width, length (in meters)
3 location 3D object location x,y,z in camera coordinates (in meters)
1 rotation_y Rotation ry around Y-axis in camera coordinates [-pi..pi]
1 score Only for results: Float, indicating confidence in detection, needed for p/r curves, higher is better.

Example:

type truncated occluded alpha bbox dimensions location rotation_y score
Pedestrian -1 -1 0.29 873.70 152.10 933.44 256.07 1.87 0.50 0.90 5.42 1.50 13.43 0.67 0.99

Citation:

If you are using this code, please cite our paper:

Joint 3D Proposal Generation and Object Detection from View Aggregation

@article{ku2017joint, title={Joint 3D Proposal Generation and Object Detection from View Aggregation}, author={Ku, Jason and Mozifian, Melissa and Lee, Jungwook and Harakeh, Ali and Waslander, Steven}, journal={arXiv preprint arXiv:1712.02294}, year={2017} }