FairMOT_distillation_knowledge

Use python 3.10.12

If you want to use the regular FairMOT version go to the snellius branch!

conda create -n MOT
conda activate MOT
conda install pytorch
cd ${FAIRMOT_ROOT}
pip install cython
pip install -r requirements.txt

Data preparation

• MOT17, MOT20 & Dancetrack MOT17 and MOT20, DanceTrack can be downloaded from the official webpage of MOT challenge. After downloading, you should prepare the data in the following structure:

MOT17
   |——————images
   |        └——————train
   |        └——————test
   └——————labels_with_ids
            └——————train(empty)
MOT20
   |——————images
   |        └——————train
   |        └——————test
   └——————labels_with_ids
            └——————train(empty)

Dancetrack
   |——————images
   |        └——————train
   |        └——————val
   └——————labels_with_ids
            └——————train(empty)
            └——————val(empty)

 Fish
   |——————images
   |        └——————train
   |        └——————val
   └——————labels_with_ids
            └——————train(empty)
            └——————val(empty)

Then, you can change the seq_root and label_root in src/gen_labels_17.py and src/gen_labels_20.py and run:

cd src
python gen_labels_17.py
python gen_labels_20.py
python gen_labels_dt.py
python gen_labels_fish.py

to generate the labels of 2DMOT15 and MOT20. The seqinfo.ini files of 2DMOT15 can be downloaded here [Google], [Baidu],code:8o0w.

Pretrained models and baseline model

• Pretrained models

HRNetV2 ImageNet pretrained model: HRNetV2-W18 official, HRNetV2-W32 official. After downloading, you should put the pretrained models in the following structure:

${FAIRMOT_ROOT}
   └——————models
           └——————hrnetv2_w32_imagenet_pretrained.pth
           └——————hrnetv2_w18_imagenet_pretrained.pth

Training

• Download the training data

• Change the dataset root directory 'root' in src/lib/cfg/data.json and 'data_dir' in src/lib/opts.py

• See snellius script folder.

• Train on MOT20: The data annotation of MOT20 is a little different from MOT17, the coordinates of the bounding boxes are all inside the image, so we need to uncomment line 313 to 316 in the dataset file src/lib/datasets/dataset/jde.py:

#np.clip(xy[:, 0], 0, width, out=xy[:, 0])
#np.clip(xy[:, 2], 0, width, out=xy[:, 2])
#np.clip(xy[:, 1], 0, height, out=xy[:, 1])
#np.clip(xy[:, 3], 0, height, out=xy[:, 3])

Tracking (evaluating)

• The default settings run tracking on the validation dataset from 2DMOT15. Using the baseline model, you can run:

See scripts in 'snellius jobs' folder.

Results of the test set all need to be evaluated on the MOT challenge server. You can see the tracking results on the training set by setting --val_motxx True and run the tracking code. We set 'conf_thres' 0.4 for MOT16 and MOT17. We set 'conf_thres' 0.3 for 2DMOT15 and MOT20.

Demo

You can input a raw video and get the demo video by running src/demo.py and get the mp4 format of the demo video:

cd src
python demo.py mot --load_model ../models/fairmot_dla34.pth --conf_thres 0.4

You can change --input-video and --output-root to get the demos of your own videos. --conf_thres can be set from 0.3 to 0.7 depending on your own videos.

Train on custom dataset

You can train FairMOT on custom dataset by following several steps bellow:

1. Generate one txt label file for one image. Each line of the txt label file represents one object. The format of the line is: "class id x_center/img_width y_center/img_height w/img_width h/img_height". You can modify src/gen_labels_16.py to generate label files for your custom dataset.
2. Generate files containing image paths. The example files are in src/data/. Some similar code can be found in src/gen_labels_crowd.py
3. Create a json file for your custom dataset in src/lib/cfg/. You need to specify the "root" and "train" keys in the json file. You can find some examples in src/lib/cfg/.
4. Add --data_cfg '../src/lib/cfg/your_dataset.json' when training.

Acknowledgement

A large part of the code is borrowed from Zhongdao/Towards-Realtime-MOT and xingyizhou/CenterNet. Thanks for their wonderful works.

Citation

@article{zhang2021fairmot,
  title={Fairmot: On the fairness of detection and re-identification in multiple object tracking},
  author={Zhang, Yifu and Wang, Chunyu and Wang, Xinggang and Zeng, Wenjun and Liu, Wenyu},
  journal={International Journal of Computer Vision},
  volume={129},
  pages={3069--3087},
  year={2021},
  publisher={Springer}
}