I noticed the quote below in the documentation, about the limitation to retrieve only the best match vector regardless of the "size" parameter in the inner_hits section of the search request.
inner_hitsfor kNN will only ever return a single hit, the nearest passage vector. Setting"size"to any value greater than1will have no effect on the results.
Currently we're doing a hybrid search, and are interested in the parent documents for the "classic" search part, and in the individual nested documents when doing knn search. The "size" parameter is ignored in the knn search regardless of whether using hybrid search or not, so the example is a simple knn search.
Is there a specific reason for this behaviour and/or will it be addressed in a future release?
It there a possibility to work around this limitation?
Using version 8.11.1
Example document:
{
"title": "First searchable document",
"paragraphs": [
{
"text": "Block of text",
"vector": [0.1, 0.2]
},
{
"text": "A second block of text",
"vector": [0.1, 0.3]
},
{
"text": "A third block of text",
"vector": [0.2, 0.3]
}
]
}
Corresponding mapping:
{
"mappings": {
"properties": {
"title": {
"type": "text"
},
"paragraphs": {
"type": "nested",
"properties": {
"text": {
"type": "text"
},
"vector": {
"type": "dense_vector",
"dims": 2,
"index": true,
"similarity": "dot_product"
}
}
}
}
}
}
Example request to be complete:
{
"fields": ["paragraphs.text"],
"_source": false,
"knn": {
"field": "paragraphs.vector",
"query_vector": [0.1234232, 0.1456789],
"num_candidates": 100,
"k": 10,
"inner_hits": {
"name": "someName",
"size": 10,
"_source": ["paragraphs.text"],
"fields": ["paragraphs.text"]
}
}
}