How to interact with rerank api endpoint

[hruday-markonda]

Hi there, I hosted a rerank model using the infinity_emb --model-name-or-path BAAI/bge-reranker-base CLI code, I am wondering how can I access this endpoint, can I access through langchain using something similar how its like in the Using Langchain with Infinity section in the Launch FAQ or maybe is there an example of a curl command to post to this endpoint?

Thank you 😊

[michaelfeil]

Thanks for your q. You can access the openapi and /docs route. Also hosting a huggingface endpoint!

[hruday-markonda]

Hi there, I am getting the following error when testing the endpoints through the \docs route using the default example. Here is the response:

{
  "error": {
    "message": "InternalServerError: the loaded moded cannot fullyfill `rerank`.options are {'embed'} inherited from model_class=<class 'infinity_emb.transformer.embedder.sentence_transformer.SentenceTransformerPatched'>",
    "type": null,
    "param": null,
    "code": 500
  }
}

How can I resolve this issue to return the reranked documents?

[michaelfeil]

Well, you need to deploy a model that is a reranker. I would recommend to run https://huggingface.co/BAAI/bge-reranker-base

[baptiste-pasquier]

I made a PR in Langchain to implement an Infinity Reranker :
langchain-ai/langchain#27043

It uses infinity_client to connect to an Infinity server.

Here is an example notebook