Deploying 3rd party HF reranking model

Elastic Search
1

Hi all,

I'm currently trying to deploy the following reranking model from Hugging Face to elasticsearch: BAAI/bge-reranker-v2-m3. I'm following this tutorial. My objective is to deploy a third-party reranking model to be able to have a deployed endpoint to either integrate into a pipeline or directly query the deployed model with a list of documents to get them reranked.

I can successfully upload the model to ES, but when I try to create the inference endpoint for the text_similarity_reranker I get an error saying: Model IDs must be unique. Requested model ID [baai__bge-reranker-base] matches existing model IDs but must not. I can confirm that the model is not deployed and there is no endpoint at this point. Here is my command for creating the inference endpoint as seen in the tutorial:

PUT _inference/rerank/bge-reranker-base
{
  "service": "elasticsearch",
  "service_settings": {
    "num_threads": 1,
    "num_allocations": 1,
    "model_id": "baai__bge-reranker-base"
  }
}

If I run the exact same command again without changing anything, I get the following error this time: Inference endpoint [bge-reranker-base] already exists but there is no deployment info for the model on the trained model page.

Any idea why this might be happening?

Current ES version is 8.15.0.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.