# Retrieve inner\_hits when searching multiple kNN fields in same nested document

**URL:** <https://discuss.elastic.co/t/retrieve-inner-hits-when-searching-multiple-knn-fields-in-same-nested-document/350024>\
**Category:** Elasticsearch\
**Tags:** vector-search\
**Created:** [December 27, 2023, 2:26pm UTC](https://discuss.elastic.co/t/retrieve-inner-hits-when-searching-multiple-knn-fields-in-same-nested-document/350024 "2023-12-27T14:26:55Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jasper\_Simon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasper_simon/32/130349_2.png) [@Jasper\_Simon](https://discuss.elastic.co/u/Jasper_Simon)\
**Post date:** [December 27, 2023, 2:26pm UTC](https://discuss.elastic.co/t/retrieve-inner-hits-when-searching-multiple-knn-fields-in-same-nested-document/350024/1 "2023-12-27T14:26:55Z")

</div>

I've got this use case (examples here are simplified to the essentials) where I want to do a knn search on multiple vectors of the same nested document inside a larger document, and be able to distinguish which of the nested documents was responsible for the hit.  
When executing the search request, an error occurs. Seems like the problem occurs when trying to combine both of these inner\_hits parts.

Does anyone know a way to make this work, or if this is a bug that is (going to be) solved in newer versions?

Below you can find the error, followed by mappings and query involved.  
Currently using `elastic v8.11.1`

```json
{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "[inner_hits] already contains an entry for key [paragraphs]"
            }
        ],
        "type": "search_phase_execution_exception",
        "reason": "all shards failed",
        "phase": "dfs_query",
        "grouped": true,
        "failed_shards": [
            {
                "shard": 0,
                "index": "aem_pages_nl_blue",
                "node": "AW2Ds3ckQnaYS9B-H34pUw",
                "reason": {
                    "type": "illegal_argument_exception",
                    "reason": "[inner_hits] already contains an entry for key [paragraphs]"
                }
            }
        ],
        "caused_by": {
            "type": "illegal_argument_exception",
            "reason": "[inner_hits] already contains an entry for key [paragraphs]",
            "caused_by": {
                "type": "illegal_argument_exception",
                "reason": "[inner_hits] already contains an entry for key [paragraphs]"
            }
        }
    },
    "status": 400
}

```

Required part of template mappings (there's more to the documents in reality):

```json
{
  "mappings": {
    "properties": {
      "websiteSection": {
        "type": "keyword"
      },
      "paragraphs": {
        "type": "nested",
        "properties": {
          "documentText": {
            "type": "text"
          },
          "documentTitle": {
            "type": "text"
          },
          "textEmbedding": {
            "type": "dense_vector",
            "dims": 1536,
            "index": true,
            "similarity": "dot_product"
          },
          "titleEmbedding": {
            "type": "dense_vector",
            "dims": 1536,
            "index": true,
            "similarity": "dot_product"
          }
        }
      }
    }
  }
}

```

The search query potentially is a hybrid query, but not in all cases. Whether it is or isn't has no effect on the result.

```json
{
  "knn": [
    {
      "field": "paragraphs.titleEmbedding",
      "query_vector": [
        "1536 floating point numbers left out for simplicity"
      ],
      "k": 20,
      "num_candidates": 50,
      "filter": [
        {
          "term": {
            "websiteSection": "forum"
          }
        }
      ],
      "inner_hits": {
        "_source": [
          "paragraphs.documentTitle"
        ],
        "fields": [
          "paragraphs.documentTitle"
        ]
      }
    },
    {
      "field": "paragraphs.textEmbedding",
      "query_vector": [
        "1536 (maybe different from previous) floating point numbers left out for simplicity"
      ],
      "k": 20,
      "num_candidates": 50,
      "filter": [
        {
          "term": {
            "websiteSection": "forum"
          }
        }
      ],
      "inner_hits": {
        "_source": [
          "paragraphs.documentText"
        ],
        "fields": [
          "paragraphs.documentText"
        ]
      }
    }
  ]
}

```

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [January 2, 2024, 2:22pm UTC](https://discuss.elastic.co/t/retrieve-inner-hits-when-searching-multiple-knn-fields-in-same-nested-document/350024/2 "2024-01-02T14:22:13Z")

</div>

@Jasper_Simon

This is 100% a bug. I am not sure of the fix yet, but I created this github issue: [Failure with `inner_hits` and multiple nested knn clauses · Issue #103792 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/issues/103792)

Thank you so much for digging into this and reporting!

---

<div class="post-metadata">

**Author:** ![BenTrent](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bentrent/32/33915_2.png) [@BenTrent](https://discuss.elastic.co/u/BenTrent)\
**Post date:** [January 2, 2024, 3:56pm UTC](https://discuss.elastic.co/t/retrieve-inner-hits-when-searching-multiple-knn-fields-in-same-nested-document/350024/3 "2024-01-02T15:56:36Z")

</div>

@Jasper_Simon actually, I think it might be OK. But I need to add some documentation.

Could you try specifying a unique `"name"` field for each of your `inner_hits` objects? Something like:

```auto
{
  "knn": [
    {
      "field": "paragraphs.titleEmbedding",
      "query_vector": [
        0.1, 0.2
      ],
      "k": 20,
      "num_candidates": 50,
      "inner_hits": {
		"name": "title",
        "_source": [
          "paragraphs.documentTitle"
        ],
        "fields": [
          "paragraphs.documentTitle"
        ]
      }
    },
    {
      "field": "paragraphs.textEmbedding",
      "query_vector": [
        0.1, 0.2
      ],
      "k": 20,
      "num_candidates": 50,
      "inner_hits": {
		"name": "text",
        "_source": [
          "paragraphs.documentText"
        ],
        "fields": [
          "paragraphs.documentText"
        ]
      }
    }
  ]
}

```

I think the issue is that we attempt to automatically name the inner\_hit to appropriately extract the correct field information given a configuration. However, we have no way of distinguishing these inner\_hit configurations.

---

<div class="post-metadata">

**Author:** ![Jasper\_Simon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasper_simon/32/130349_2.png) [@Jasper\_Simon](https://discuss.elastic.co/u/Jasper_Simon)\
**Post date:** [January 3, 2024, 8:20am UTC](https://discuss.elastic.co/t/retrieve-inner-hits-when-searching-multiple-knn-fields-in-same-nested-document/350024/4 "2024-01-03T08:20:42Z")

</div>

Hi @BenTrent, thanks a lot for your solution. It is simple yet effective!

Another thing I noticed in the documentation that is currently holding us back, is the limitation to retrieve only the best match vector regardless of the "size" parameter in the inner\_hits.

I made a [separate post](https://discuss.elastic.co/t/size-parameter-ignored-in-nested-knn-search/350285) about that in order to not clutter this page.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 31, 2024, 8:21am UTC](https://discuss.elastic.co/t/retrieve-inner-hits-when-searching-multiple-knn-fields-in-same-nested-document/350024/5 "2024-01-31T08:21:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.