Skip to content

Latest commit

 

History

History
23 lines (13 loc) · 782 Bytes

README.md

File metadata and controls

23 lines (13 loc) · 782 Bytes
Raw

Evaluate your reranker quickly!

Usage

Install Pyserini and ColBERT, please refer to the original installation guide.

Run the following command to evaluate your reranker:

python eval_reranker.py --dataset $dataset --reranker $model

Supported datasets are listed in the file. Now, the eval code only supports BM25 as the first-stage retriever and only supports ColBERT and cross-encoder as the reranker.

Results

Coming soon!

Acknowledgement

  • RankGPT: The code is mainly modified from this repository.
  • Pyserini: Thanks for providing the pre-build indexes of MSMARCO and BEIR!