# \[v8.17.1\] Bug in Semantic Reranking using Vertex AI?

**URL:** <https://discuss.elastic.co/t/v8-17-1-bug-in-semantic-reranking-using-vertex-ai/380389>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning\
**Created:** [July 23, 2025, 4:05pm UTC](https://discuss.elastic.co/t/v8-17-1-bug-in-semantic-reranking-using-vertex-ai/380389 "2025-07-23T16:05:24Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ik-southpole](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ik-southpole/32/144292_2.png) [@ik-southpole](https://discuss.elastic.co/u/ik-southpole)\
**Post date:** [July 23, 2025, 4:05pm UTC](https://discuss.elastic.co/t/v8-17-1-bug-in-semantic-reranking-using-vertex-ai/380389/1 "2025-07-23T16:05:24Z")

</div>

Good day,

I am implementing a semantic ranking in v8.17.1 using Vertex AI (model `semantic-ranker-default@latest`) and I am facing some potentially strange behaviour.

## Context

I followed [Semantic reranking in Elasticsearch with retrievers](https://www.elastic.co/search-labs/blog/semantic-reranking-with-retrievers).

I've created the inference endpoint `google_vertex_ai_rerank`. I sent the following request to verify it was working.

```auto
POST _inference/rerank/google_vertex_ai_rerank
{
  "query": "What is the capital of the USA?",
  "input": [
    "Carson City is the capital city of the American state of Nevada. At the 2010 United States Census, Carson City had a population of 55,274.",
    "Capital punishment (the death penalty) has existed in the United States since before the United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states.",
    "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean that are a political division controlled by the United States. Its capital is Saipan.",
    "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district.",
    "Charlotte Amalie is the capital and largest city of the United States Virgin Islands. It has about 20,000 people. The city is on the island of Saint Thomas.",
    "North Dakota is a state in the United States. 672,591 people lived in North Dakota in the year 2010. The capital and seat of government is Bismarck."
  ]
}

```

Which returned the following response:

```jsonc
{
  "rerank": [
    {
      "index": 0, // <- Index must be 3
      "relevance_score": 0.9375,
      "text": "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district."
    },
    {
      "index": 1, // <- Wrong index
      "relevance_score": 0.142,
      "text": "Carson City is the capital city of the American state of Nevada. At the 2010 United States Census, Carson City had a population of 55,274."
    },
    {
      "index": 2, // <- Wrong index
      "relevance_score": 0.1122,
      "text": "North Dakota is a state in the United States. 672,591 people lived in North Dakota in the year 2010. The capital and seat of government is Bismarck."
    },
    {
      "index": 3, // <- Wrong index
      "relevance_score": 0.0689,
      "text": "Charlotte Amalie is the capital and largest city of the United States Virgin Islands. It has about 20,000 people. The city is on the island of Saint Thomas."
    },
    {
      "index": 4, // <- Wrong index
      "relevance_score": 0.0551,
      "text": "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean that are a political division controlled by the United States. Its capital is Saipan."
    },
    {
      "index": 5, // <- Wrong index
      "relevance_score": 0.0476,
      "text": "Capital punishment (the death penalty) has existed in the United States since before the United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states."
    }
  ]
}

```

## Problem

The result has ranked records with incorrect `index`.

I expected the following response:

```json
{
  "rerank": [
    {
      "index": 3,
      "relevance_score": 0.9375,
      "text": "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district."
    }
    // ... Truncated 
  ]
}

```

As a result, when I use `text_similarity_reranker` the order of documents is wrong.

Should I report a problem to Google Vertex team?  
Am I doing something wrong?

Thank you in advance!

---

<div class="post-metadata">

**Author:** ![dkyle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dkyle/32/59114_2.png) [@dkyle](https://discuss.elastic.co/u/dkyle)\
**Post date:** [July 29, 2025, 8:54pm UTC](https://discuss.elastic.co/t/v8-17-1-bug-in-semantic-reranking-using-vertex-ai/380389/2 "2025-07-29T20:54:52Z")

</div>

Hi @ik-southpole, the bug is in Elasticsearch and fixed in version 8.18.0. Sorry about this, please upgrade to 8.18 to pick up the fix.

> <https://github.com/elastic/elasticsearch/pull/117287>
>
> For the Google Vertex AI rerank API, for an infer request with the following exa…mple inputs:
> 
> \`\`\`
> {
> "input": \[
> "A canvas stretched across the day,\\nWhere sunlight learns to dance and play.\\nBlue, a hue of scattered light,\\nA gentle whisper, soft and bright.",
> "The sky appears blue due to a phenomenon called Rayleigh scattering. Sunlight is comprised of all the colors of the rainbow. Blue light has shorter wavelengths than other colors, and is thus scattered more easily."
> \]
> }
> \`\`\`
> 
> We create a request that looks like this, following the \[API specs\](https://cloud.google.com/generative-ai-app-builder/docs/ranking)
> \`\`\`
> "records": \[
> {
> "id": "1",
> "content": "A canvas stretched across the day,\\nWhere sunlight learns to dance and play.\\nBlue, a hue of scattered light,\\nA gentle whisper, soft and bright."
> },
> {
> "id": "2",
> "content": "The sky appears blue due to a phenomenon called Rayleigh scattering. Sunlight is comprised of all the colors of the rainbow. Blue light has shorter wavelengths than other colors, and is thus scattered more easily."
> }
> \]
> \`\`\`
> Note that for the ID, we use the index of the entry in the input array https://github.com/elastic/elasticsearch/blob/6d963d324aa71a514cd4baa01bd0805eaba7426e/x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/external/request/googlevertexai/GoogleVertexAiRerankRequestEntity.java#L47-L53
> 
> The API response is an array of records, sorted in descending order by score, for example
> 
> \`\`\`
> {
> "records": \[
> {
> "id": "2",
> "score": 0.98,
> "content": "The sky appears blue due to a phenomenon called Rayleigh scattering. Sunlight is comprised of all the colors of the rainbow. Blue light has shorter wavelengths than other colors, and is thus scattered more easily."
> },
> {
> "id": "1",
> "score": 0.64,
> "content": "A canvas stretched across the day,\\nWhere sunlight learns to dance and play.\\nBlue, a hue of scattered light,\\nA gentle whisper, soft and bright."
> }
> \]
> }
> \`\`\`
> 
> In our response parser, we were not using the record ID (index of the entry in the input array) as the index into our result array, resulting in results where the score did not match up to the entry in the input array.

---

<div class="post-metadata">

**Author:** ![ik-southpole](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ik-southpole/32/144292_2.png) [@ik-southpole](https://discuss.elastic.co/u/ik-southpole)\
**Post date:** [July 30, 2025, 7:53am UTC](https://discuss.elastic.co/t/v8-17-1-bug-in-semantic-reranking-using-vertex-ai/380389/3 "2025-07-30T07:53:21Z")

</div>

Thank you for your hint! It worked!