VA-Count

[ECCV 2024] Zero-shot Object Counting with Good Exemplars [paper]

Zero-shot Object Counting with Good Exemplars

News🚀

• 2024.09.27: Our code is released.
• 2024.09.26: Our inference code has been updated, and the code for selecting exemplars and the training code will be coming soon.
• 2024.07.02: VA-Count is accepted by ECCV2024.

Overview

Overview of the proposed method. The proposed method focuses on two main elements: the Exemplar Enhancement Module (EEM) for improving exemplar quality through a patch selection integrated with Grounding DINO, and the Noise Suppression Module (NSM) that distinguishes between positive and negative class samples using density maps. It employs a Contrastive Loss function to refine the precision in identifying target class objects from others in an image.

Environment

pip install torch==1.10.0+cu111 torchvision==0.11.0+cu111 torchaudio==0.10.0 -f https://download.pytorch.org/whl/torch_stable.html
pip install timm==0.3.2
pip install numpy
pip install matplotlib tqdm 
pip install tensorboard
pip install scipy
pip install imgaug
pip install opencv-python
pip3 install hub

For more information on Grounding DINO, please refer to the following link:

GroundingDINO We are very grateful for the Grounding DINO approach, which has been instrumental in our work！

Datasets

• FSC147

• CARPK

Preparing the datasets as follows:

./data/
|--FSC147
|  |--images_384_VarV2
|  |  |--2.jpg
|  |  |--3.jpg
|  |--gt_density_map_adaptive_384_VarV2
|  |  |--2.npy
|  |  |--3.npy
|  |--annotation_FSC147_384.json
|  |--Train_Test_Val_FSC_147.json
|  |--ImageClasses_FSC147.txt
|  |--train.txt
|  |--test.txt
|  |--val.txt
|--CARPK/
|  |--Annotations/
|  |--Images/
|  |--ImageSets/

Inference

• For inference, you can download the model from Baidu-Disk, passward:paeh

python FSC_test.py --output_dir ./data/out/results_base --resume ./data/checkpoint_FSC.pth

Single and Multiple Object Classifier Training

python datasetmake.py
python biclassify.py

• You can also directly download the model from Baidu-Disk, passward:psum Save it in ./data/out/classify/

Generate exemplars

python grounding_pos.py --root_path ./data/FSC147/
python grounding_neg.py --root_path ./data/FSC147/

Train

CUDA_VISIBLE_DEVICES=0 python FSC_pretrain.py \
    --epochs 500 \
    --warmup_epochs 10 \
    --blr 1.5e-4 --weight_decay 0.05

• You can also directly download the pre-train model from Baidu-Disk, passward:xynw Save it in ./data/

CUDA_VISIBLE_DEVICES=0 python FSC_train.py --epochs 1000 --batch_size 8 --lr 1e-5 --output_dir ./data/out/

Citation

@inproceedings{zhu2024zero,
  title={Zero-shot Object Counting with Good Exemplars},
  author={Zhu, Huilin and Yuan, Jingling and Yang, Zhengwei and Guo, Yu and Wang, Zheng and Zhong, Xian and He, Shengfeng},
  booktitle={Proceedings of the European Conference on Computer Vision},
  year={2024}
}

Acknowledgement

This project is based on the implementation from CounTR, we are very grateful for this work and GroundingDINO.

If you have any questions, please get in touch with me (jsj_zhl@whut.edu.cn).