❗️ FlashRankRanker ranking is incorrect – it returns the documents in input order

[lsorber]

The ranking produced by FlashRankRanker is incorrect. It returns the documents in input order.

This statement in flashrank_ranker.py¹

        ranked_results = [
            Result(
                document=docs[idx],  # Returns documents in input order.
                score=result["score"],
                rank=idx + 1,
            )
            for idx, result in enumerate(flashrank_results)
        ]

should be:

        ranked_results = [
            Result(
                document=docs[result["id"]],  # Returns reranked documents.
                score=result["score"],
                rank=idx + 1,
            )
            for idx, result in enumerate(flashrank_results)
        ]

Footnotes

1. https://github.com/AnswerDotAI/rerankers/blob/c5b2c19e7a84fc4c4dd3f31dea934297ce4b7bb1/rerankers/models/flashrank_ranker.py#L48 ↩