Skip to content

Latest commit

 

History

History
44 lines (29 loc) · 4.83 KB

DETECTOR.md

File metadata and controls

44 lines (29 loc) · 4.83 KB
Raw

Fine-tuning object detectors

How to use your own detector

NOTE: If you wish the use the same detector for a fair comparison, you can replace the provided fine-tuned detector with your own detector with the following steps:

  1. Generate a detection file in .pkl, following the iCAN format here.
  2. Update the path of your detector file in default config here and here.

Results

We here provide some numbers of our method by replacing the object detector. Note that we simply replace the detector file without re-training our method.

Detector Obj det. mAP on HICO test Full (python) Rare (python) Non Rare (python) Full (Matlab) Rare (Matlab) Non Rare (Matlab)
Our fine-tuned - 24.51 19.4 26.03 24.53 19.47 26.04
R101-FPN Faster R-CNN (from VCL) [ckpt] [pkl] 30.79 20.71 16.37 22.01 - - -
R50 DETR (from UPT)[ckpt] [pkl] - 21.52 19.31 22.19 - - -
R101 DETR (from UPT)[ckpt] [pkl] - 21.98 19.5 22.72 - - -
R101-DC5 DETR (from UPT)[ckpt] [pkl] - 21.66 19.66 22.25 - - -
*X152-FPN Cascade R-CNN [ckpt] [pkl] 37.61 24.89 20.2 26.28 - - -

*We re-train a model for reproducibility only, please use our original detection pkl file if you wish to compare to other methods using the same detector.

Fine-tuning w/ detectron2

  1. Set up your detectron2 environment following the official instructions.
  2. Download HICO detection annotations from VCL: [training set] [test set], and register a new dataset accordingly in detectron2.
  3. Choose a config file and download the COCO pre-trained weights, then you can start. Usually a learning rate 1e-3 is good.
  4. To reproduce the new X152-FPN Cascade R-CNN results, you can use this config file and download the pre-trained weight. We trained it using 2 V100 GPUs and take the 20k step checkpoint. Note that it has not converged, so you may get even better results if you select a later checkpoint.

We also provide a simple script to convert detectron2 test json file to iCAN format pkl file. Please find it here.

Fine-tuning w/ DETR

Set up a forked version of DETR here. You will need to follow the instructions to install the pocket library.

To get iCAN format pkl file from this repo:

  1. Replace the original pocket/data/hicodet.py in the installed pocket directory, with this file. The original pocket implementation skips ~100 images in the test set, while our current python evaluation requires evaluating on all the test images even if they have no annotations.
  2. Test your model and output iCAN format pkl file using this script.

Conclusion

Object detector can largely affect the final HOI detection results. Sometimes a 0.02 difference in object detection mAP can lead to 0.3~0.4 difference in HOI detection. Thus, we highly recommend using the same detector for a fair comparison. If you use your own fine-tuned detector, please try to use it for your competing methods as well.