This is the official repository of Simple Cues Lead to a Strong Multi-Object Tracker.
To set up this repository follow the following steps
git clone https://github.com/dvl-tum/GHOST.git
cd GHOST
git clone https://github.com/dvl-tum/TrackEvalForGHOST.git
Download anaconcda and create conda evironment using env_from_history.yml file by:
conda env create -f env_from_history.yml
Then activate the environment using:
conda activate GHOST
Download MOT17, MOT20, and DanceTrack tracking datasets. For BDD100k download MOT 2020 Labels and MOT 2020 images. Unzip all of them to datasets.
Finally also download our detections used and also extract into dataset. For MOT17 we also provide the bounding boxes from various trackers on the validation set, i.e., first half of all training sequences.
The final data structure should look like the following:
datasets/
- bdd100k
- images
- track
- train
- val
- test
- labels
- box_track_20
- train
- val
- DanceTrack
- train
- val
- test
- MOT17
- train
- test
- MOT20
- train
- test
- detections_GHOST
- bdd100k
- train
- val
- test
- DanceTrack
- val
- test
- MOT17
- train
- test
- MOT20
- train
- test
Download our pretrained ReID weights and extract them into ReID/trained_models so that in the end the data structure looks like the the following:
ReID/
* trained_models
* market_models
* resnet50_Market.pth
* MOT_models
* split1_resnet50_MOT17.pth
* split2_resnet50_MOT17.pth
* split3_resnet50_MOT17.pth
The final model that we use is market_models/resnet50_Market.pth.
To re-train the ReID model on Market-1501 or MOT17, we provide the dataset structures we used for Market and MOT17. Please unzip them into the following structure:
ReID/
* datasets
* MOT17_ReID
* Market-1501-v15.09.15
To run our tracker run on MOT17 private detections run:
bash scripts/main_17.sh
and to run in with public center track preprocessed detections run:
bash scripts/main_17_pub.sh
Similarly, you can find scripts for main_20.sh, main_20_pub.sh using tracktor preprocessed detections, main_dance.sh, and main_bdd.sh in the scripts directory.
You can define the following parameters directly in the bash file:
| Parameter | description |
|---|---|
| --config_path | Path to config file |
| --det_conf | Minimum detection confidence |
| --act | Matching threshold for active tracks |
| --inact | Matching threshold for inactive tracks |
| --det_file | Detections to be used (see dataset/detections_GHOST for names) |
| --only_pedestrian | If only pedestrian class should be used for evaluation |
| --inact_patience | Patience for inactive tracks to be used during tracking |
| --combi | How to combine motion and appearance distance (sum_0.3 means weighted sum with motion weight 0.3) |
| --store_feats | Store features for analysis |
| --on_the_fly | If using on the fly domain adaptation |
| --do_inact | If using proxy distance / proxy feature compuatation for inactive tracks |
| --splits | Which split to use (see data/splits.py for different splits) |
| --len_thresh | Minimum length of tracks (default set to 0) |
| --new_track_conf | Confidence threshold for detection to start new track |
| --remove_unconfirmed | If removing unconfirmed tracks (tracks that are only initialized and then no detection added in next frame, default is 0) |
| --last_n_frames | Number of last frames used to compute velocity for motion model |
For others, like data paths, please refer directly to the config files in config/.
For the test submissions please adapt the split in the configuration parameters to the corresponding splits (data/splits.py).
For submission please zip the files in the corresponding output directories and submit to the test servers of MOT17, MOT20, DanceTrack.
If you want to submit to BDD server, please utilize the corresponding experiment directory in the bdd_for_submission directory that is directly generated, zip the files directly (not the directory), and upload under Submit to the server.
If you want to use different distance computations than the current proxy distance computation, you have to change the avg_act and avg_inact sections in the config files the following for other proxy distances:
| Do | num | proxy | Description |
|---|---|---|---|
| 1 | 1 | 'each_sample' | Min of distances between features of new detection and features of all prior detections in track |
| 1 | 2 | 'each_sample' | Mean of distances between features of new detection and features of all prior detections in track |
| 1 | 3 | 'each_sample' | Max of distances between features of new detection and features of all prior detections in track |
| 1 | 4 | 'each_sample' | (Min + Max)/2 of distances between features of new detection and features of all prior detections in track |
| 1 | 5 | 'each_sample' | Median of distances between features of new detection and features of all prior detections in track |
| 1 | x | 'first' | Uses the features of the first detection in the track for distance computation with the features of the new detection, x does not matter |
| 1 | x | 'last' | Uses the features of the last detection in the track for distance computation with the features of the new detection, x does not matter |
| 1 | x | 'mv_avg' | Uses the moving average of features in the track for distance computation with the features of the new detection, x is the update weight |
| 1 | x | 'mean' | Uses the mean of features in the track for distance computation with the features of the new detection, x number of last detections to be used |
| 1 | x | 'median' | Uses the median of features in the track for distance computation with the features of the new detection, x number of last detections to be used |
| 1 | x | 'mode' | Uses the mode of features in the track for distance computation with the features of the new detection, x number of last detections to be used |
If you set do to 0 it falls back to using the features of the last detection in a track.
If you want to run the histogram or quantile analysis as in the paper please first run an experiment on the detection set you want to use. The features will be stored to features. Then run the
python tools/investigate_features.py
the corresponding figures will be stored to histograms_features.