Size parameter ignored in nested knn search

Elastic Stack Elasticsearch
1

I noticed the quote below in the documentation, about the limitation to retrieve only the best match vector regardless of the "size" parameter in the inner_hits section of the search request.

inner_hits for kNN will only ever return a single hit, the nearest passage vector. Setting "size" to any value greater than 1 will have no effect on the results.

Currently we're doing a hybrid search, and are interested in the parent documents for the "classic" search part, and in the individual nested documents when doing knn search. The "size" parameter is ignored in the knn search regardless of whether using hybrid search or not, so the example is a simple knn search.

Is there a specific reason for this behaviour and/or will it be addressed in a future release?
It there a possibility to work around this limitation?

Using version 8.11.1

Example document:

{
  "title": "First searchable document",
  "paragraphs": [
    {
      "text": "Block of text",
      "vector": [0.1, 0.2]
    },
    {
      "text": "A second block of text",
      "vector": [0.1, 0.3]
    },
    {
      "text": "A third block of text",
      "vector": [0.2, 0.3]
    }
  ]
}

Corresponding mapping:

{
  "mappings": {
    "properties": {
      "title": {
        "type": "text"
      },
      "paragraphs": {
        "type": "nested",
        "properties": {
          "text": {
            "type": "text"
          },
          "vector": {
            "type": "dense_vector",
            "dims": 2,
            "index": true,
            "similarity": "dot_product"
          }
        }
      }
    }
  }
}

Example request to be complete:

{
  "fields": ["paragraphs.text"],
  "_source": false,
  "knn": {
    "field": "paragraphs.vector",
    "query_vector": [0.1234232, 0.1456789],
    "num_candidates": 100,
    "k": 10,
    "inner_hits": {
      "name": "someName",
      "size": 10,
      "_source": ["paragraphs.text"],
      "fields": ["paragraphs.text"]
    }
  }
}
2

@Jasper_Simon that is indeed a current limitation. You can track the feature progress here: [Feature Request] Support inner_hits.size > 1 for nested dense vector fields · Issue #102950 · elastic/elasticsearch · GitHub

3

@BenTrent I see that the feature request has been merged - awesome, thank you for that!
In which release version should I expect this to start working?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.