CBIR in academia and industry

Awesome image retrieval papers

The main goal is to collect classical and solid works of image retrieval in academia and industry.

• Classical Local Feature
• Deep Learning Feature (Global Feature)
• Deep Learning Feature (Local Feature)
• Deep Learning Feature (Instance Search)
• ANN search
• CBIR Attack
• CBIR rank
• CBIR in Industry
• CBIR Competition and Challenge
• CBIR for Duplicate(copy) detection
• Feature Fusion
• Instance Matching
• Semantic Matching
• Template Matching
• Image Identification
• Tutorials
• Slide
• Demo and Demo Online
• Datasets
• Useful Package

Classical Local Feature

• Object retrieval with large vocabularies and fast spatial matching, CVPR 2007.
• Visual Categorization with Bags of Keypoints, ECCV 2004.
• ORB: an efficient alternative to SIFT or SURF, ICCV 2011.
• Object Recognition from Local Scale-Invariant Features, ICCV 1999.
• Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval, ICCV 2007.
• Three things everyone should know to improve object retrieval, CVPR 2012.
• On-the-fly learning for visual search of large-scale image and video datasets
• All about VLAD, CVPR 2013.
• Aggregating localdescriptors into a compact image representation, CVPR 2010.
• More About VLAD: A Leap from Euclidean to Riemannian Manifolds, CVPR 2015.
• Hamming embedding and weak geometric consistency for large scale image search, CVPR 2008.
• Revisiting the VLAD image representation, project
• Improving the Fisher Kernel for Large-Scale Image Classification, ECCV 2010.
• Image Classification with the Fisher Vector: Theory and Practice
• Democratic Diffusion Aggregation for ImageRetrieval
• A Vote-and-Verify Strategy for Fast Spatial Verification in Image Retrieval, ACCV 2016.
• Triangulation embedding and democratic aggregation for image search, CVPR 2014.
• Efficient Large-scale Image Search With a Vocabulary Tree, IPOL 2015, code.

Deep Learning Feature (Global Feature)

• Online Invariance Selection for Local Feature Descriptors, ECCV 2020, code.
• Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval, ECCV 2020.
• SOLAR: Second-Order Loss and Attention for Image Retrieval, ECCV 2020.
• Unifying Deep Local and Global Features for Image Search, arxiv 2020.
• SOLAR: Second-Order Loss and Attention for Image Retrieval, arxiv 2020.
• A Benchmark on Tricks for Large-scale Image Retrieval，arxiv 2020.
• Learning with Average Precision: Training Image Retrieval with a Listwise Loss, ICCV 2019.
• MultiGrain: a unified image embedding for classes and instances, arxiv 2019.
• Deep Image Retrieval:Learning Global Representations for Image search.
• End-to-end Learning of Deep Visual Representations for Image retrieval, DIR更详细的论文说明.
• What Is the Best Practice for CNNs Applied to Visual Instance Retrieval?, 关于layer选取的问题.
• Bags of Local Convolutional Features for Scalable Instance Search.
• Faster R-CNN Features for Instance Search, CVPR workshop 2016.
• Cross-dimensional Weighting for Aggregated Deep Convolutional Features, project.
• Class-Weighted Convolutional Features for Image Retrieval.
• Multi-Scale Orderless Pooling of Deep Convolutional Activation Features, VLAD coding.
• Aggregating Deep Convolutional Features for Image Retrieval, 论文笔记, 基于深度学习的视觉实例搜索研究进展.
• Particular object retrieval with integral max-pooling of CNN activations, project.
• Particular object retrieval using CNN.
• Learning to Match Aerial Images with Deep Attentive Architectures.
• Siamese Network of Deep Fisher-Vector Descriptors for Image Retrieval.
• Combining Fisher Vector and Convolutional Neural Networks for Image Retrieval, fv和cnn特征融合提升.
• Selective Deep Convolutional Features for Image Retrieval, ACM MM 2017.
• Class-Weighted Convolutional Features for Image Retrieval.
• Fine-tuning CNN Image Retrieval with No Human Annotation, TPAMI 2018.
• An accurate retrieval through R-MAC+ descriptors for landmark recognition.
• Regional Attention Based Deep Feature for Image Retrieval, code, BMVC 2018.
• Detect-to-Retrieve: Efficient Regional Aggregation for Image Search, CVPR 2019.
• Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking, project, CVPR 2018.
• Guided Similarity Separation for Image Retrieval, NeurIPS 2019.

Deep Learning Feature (Local Feature)

• Glue Factory is CVG's library for training and evaluating deep neural network that extract and match local visual feature, code
• DeDoDe: Detect, Don't Describe -- Describe, Don't Detect for Local Feature Matching, arXiv 2023, code.
• LightGlue: Local Feature Matching at Light Speed, arXiv 2023, code.
• Simple Learned Keypoints, a self-supervised deep learning keypoint model, arxiv 2023, code.
• Learning Super-Features for Image Retrieval, ICLR 2022, code.
• LoFTR: Detector-Free Local Feature Matching with Transformers, CVPR 2021, code.
• DFM: A Performance Baseline for Deep Feature Matching, CVPRW 2021, code.
• COTR: Correspondence Transformer for Matching Across Images, arxiv 2021.
• Online Invariance Selection for Local Feature Descriptors, ECCV 2020, code.
• Learning and aggregating deep local descriptors for instance-level recognition, ECCV 2020, code.
• DISK: Learning local features with policy gradient, NeurIPS 2020, code.
• Learning and aggregating deep local descriptorsfor instance-level recognition, ECCV 2020, code.
• D2D: Keypoint Extraction with Describe to Detect Approach, arxiv 2020.
• UR2KiD: Unifying Retrieval, Keypoint Detection, and Keypoint Description without Local Correspondence Supervision, arxiv.
• Visualizing Deep Similarity Networks, WACV 2019.
• Combination of Multiple Global Descriptors for Image Retrieval.
• Beyond Cartesian Representations for Local Descriptors, code, ICCV 2019.
• R2D2: Reliable and Repeatable Detector and Descriptor, code, NeurIPS 2019.
• SOSNet: Second Order Similarity Regularization for Local Descriptor Learning, CVPR 2019.
• Local Features and Visual Words Emerge in Activations, CVPR 2019.
• Explicit Spatial Encoding for Deep Local Descriptors, CVPR 2019.
• Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters, ICCV 2019.
• Learning Discriminative Affine Regions via Discriminability, affnet.
• A Large Dataset for Improving Patch Matching, PS-Dataset.
• Working hard to know your neighbor's margins: Local descriptor learning loss, code.
• MatchNet: Unifying Feature and Metric Learning for Patch-Based Matching, code.
• LF-Net: Learning Local Features from Images, NeurIPS 2018.
• Local Descriptors Optimized for Average Precision, CVPR 2018.
• SuperPoint: Self-Supervised Interest Point Detection and Description, Magic Leap.
• GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints, code, ECCV 2018.
• Learning local feature descriptors with triplets and shallow convolutional neural networks, BMVC 2016.

Deep Learning Feature (Instance Search)

• Deeply Activated Salient Region for Instance Search, arXiv 2020.
• Instance search based on weakly supervised feature learning, Neurocomputing 2019.
• Instance Search via Instance Level Segmentation and Feature Representation, arXiv 2018.
• Unsupervised object discovery for instance recognition, WACV 2018.
• Faster R-CNN Features for Instance Search, CVPR workshop 2016.

ANN search

• Results of the NeurIPS’21 Challenge on Billion-Scale Approximate Nearest Neighbor Search.
• Nearest neighbor search with compact codes: A decoder perspective, arxiv 2021.
• Accelerating Large-Scale Inference with Anisotropic Vector Quantization, blog, code, ICML 2020.
• Improving Approximate Nearest Neighbor Search through Learned Adaptive Early Termination, SIGMOD 2020.
• RobustiQ A Robust ANN Search Method for Billion-scale Similarity Search on GPUs, ICMR 2019.
• Zoom: Multi-View Vector Search for Optimizing Accuracy, Latency and Memory.
• Vector and Line Quantization for Billion-scale Similarity Search on GPUs.
• GGNN: Graph-based GPU Nearest Neighbor Search, arxiv 2019, code.
• Learning to Route in Similarity Graphs, ICML 2019.
• Practical and Optimal LSH for Angular Distance.
• pq-fast-scan.
• faiss. A library for efficient similarity search and clustering of dense vectors.
• Polysemous codes.
• Optimized Product Quantization.
• lopq. Training of Locally Optimized Product Quantization (LOPQ) models for approximate nearest neighbor search of high dimensional data in Python and Spark.
• nns_benchmark. Benchmark of Nearest Neighbor Search on High Dimensional Data.
• Optimized Product Quantization.
• Falconn. FAst Lookups of Cosine and Other Nearest Neighbors.
• Annoy. Approximate Nearest Neighbors in C++/Python optimized for memory usage and loading/saving to disk.
• NMSLIB. Non-Metric Space Library (NMSLIB): A similarity search library and a toolkit for evaluation of k-NN methods for generic non-metric spaces.
• Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs, graph-based method.
• Fast Approximate Nearest Neighbor Search With Navigating Spreading-out Graphs, code
• Efficient Nearest Neighbors Search for Large-Scale Landmark Recognition
• NV-tree: A Scalable Disk-Based High-Dimensional Index.
• Dynamicity and Durability in Scalable Visual Instance Search.
• Revisiting the Inverted Indices for Billion-Scale Approximate Nearest Neighbors，code.
• Link and code: Fast indexing with graphs and compact regression codes.
• A Survey of Product Quantization，对于矢量量化方法一篇比较完整的调研，值得一读.
• GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints，学习局部特征的descriptor，匹配能力较强.
• Learning a Complete Image Indexing Pipeline, CVPR 2018.
• spreading vectors for similarity search, ICLR 2019.
• SPTAG: A library for fast approximate nearest neighbor search. Microsoft.

CBIR Attack

• Open Set Adversarial Examples.

CBIR rank

• Fast Spectral Ranking for Similarity Search, code, CVPR 2018.

CBIR in Industry

• Videntifier is a visual search engine based on a patented large-scale local feature database, demo, based on SIFT feature and NV-tree. (Chinese blog post).
• Web-Scale Responsive Visual Search at Bing.
• Visual Search at Alibaba.
• Visual Search at Pinterest.
• Visual Discovery at Pinterest.
• Learning a Unified Embedding for Visual Search at Pinterest, KDD 2019.
• Visual Search at ebay.
• Deep Learning based Large Scale Visual Recommendation and Search for E-Commerce, project.
• 微信「扫一扫识物」 的背后技术揭秘.
• 揭秘微信「扫一扫」识物为什么这么快？

CBIR Competition and Challenge

• The 2021 Image Similarity Dataset and Challenge, 2021, code.
• Google Landmark Retrieval Challenge, 2018.
• Alibaba Large-scale Image Search Challenge, 2015.
• Pkbigdata image retrieval, 2015.
• Large-scale Landmark Retrieval/Recognition under a Noisy and Diverse Dataset, Landmark2019-1st-and-3rd-Place-Solution.

CBIR for Duplicate(copy) detection

• A Self-Supervised Descriptor for Image Copy Detection, CVPR 2022, code.
• A Robust and Fast Video Copy Detection System Using Content-Based Fingerprinting.

Feature Fusion

• Feature fusion using Canonical Correlation Analysis.

Instance Matching

• Neural- Guided RANSAC: Learning Where to Sample Model Hypotheses, ICCV 2019, code.
• AdaLAM: Revisiting Handcrafted Outlier Detection, arxiv 2006.
• Graph-Cut RANSAC, code
• Image Matching Benchmark
• GMS: Grid-based Motion Statistics for Fast, Ultra-robust Feature Correspondence
• A Vote-and-Verify Strategy for Fast Spatial Verification in Image Retrieval
• CODE: Coherence Based Decision Boundaries for Feature Correspondence
• Robust feature matching in 2.3µs
• PopSift is an implementation of the SIFT algorithm in CUDA
• openMVG robust_estimation
• Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses.
• Homography from two orientation- and scale-covariant features, code.

Semantic Matching

• End-to-end weakly-supervised semantic alignment.

Template Matching

• QATM: Quality-Aware Template Matching For Deep Learning, CVPR 2019.

Image Identification

• Image Identification Using SIFT Algorithm: Performance Analysis against Different Image Deformations.

Tutorials

• PyRetri, Open source deep learning based image retrieval toolbox based on PyTorch.
• How to Apply Distance Metric Learning to Street-to-Shop Problem.
• Recent Image Search Techniques.
• Compact Features for Visual Search.
• multimedia-indexing. A framework for large-scale feature extraction, indexing and retrieval.
• Image Similarity using Deep Ranking, code.
• Triplet Loss and Online Triplet Mining in TensorFlow.
• tf_retrieval_baseline.

Slide

• VRG Prague in “Large-Scale Landmark Recognition Challenge”, ranked 3rd in the Google Landmark Recognition Challenge.

Demo and Demo Online

• Visual Image Retrieval and Localization, SIFT feature encoded by BOW.
• VGG Image Search Engine, SIFT feature encoded by BOW.
• SoTu, A flask-based cbir system.
• yisou, A flask-based painting cbir system, the search algorithm is designed by Yong Yuan.

Datasets

• DeepFashion2 Dataset, DeepFashion2 is a comprehensive fashion dataset.
• Holidays, Holidays consists images from personal holiday albums of various scene types.
• Oxford, Oxford consists of 11 different Oxford landmarks.
• Paris, Paris consists of images crawled from 11 queries on specific Paris architecture.
• ROxford and RParis, ROxford and RParis are revisited versions of the original Oxford and Paris with annotation corrections, enlarged sizes and more difficult samples.
• INSTRE, INSTRE is an instance-level object retrieval dataset.

Useful Package

• VLFeat
• Yael

Star History