# Is it possible to eliminate duplication of search response when using nested query?

**URL:** <https://discuss.elastic.co/t/is-it-possible-to-eliminate-duplication-of-search-response-when-using-nested-query/169750>\
**Category:** Elasticsearch\
**Created:** [February 25, 2019, 4:52am UTC](https://discuss.elastic.co/t/is-it-possible-to-eliminate-duplication-of-search-response-when-using-nested-query/169750 "2019-02-25T04:52:34Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Masanori\_Ohnishi](https://avatars.discourse-cdn.com/v4/letter/m/278dde/32.png) [@Masanori\_Ohnishi](https://discuss.elastic.co/u/Masanori_Ohnishi)\
**Post date:** [February 25, 2019, 4:52am UTC](https://discuss.elastic.co/t/is-it-possible-to-eliminate-duplication-of-search-response-when-using-nested-query/169750/1 "2019-02-25T04:52:34Z")

</div>

Here's the sample of mapping, register, and search query.

## mapping

```auto
curl -X PUT "es:9200/english" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "_doc": {
      "properties": {
        "title" : {
          "type" : "text"
        },
        "contents": {
          "type": "nested"
        }
      }
    }
  }
}
'

```

## register

```auto
curl -X PUT "es:9200/english/_doc/1?refresh" -H 'Content-Type: application/json' -d'
{
  "title": "Test title",
  "contents": [
    {
      "header": "something special",
      "body": "I am John."
    },
    {
      "header": "anything hot",
      "body": "This is a cup."
    }
  ]
}
'

curl -X PUT "es:9200/english/_doc/2?refresh" -H 'Content-Type: application/json' -d'
{
  "title": "Test title",
  "contents": [
    {
      "header": "something special",
      "body": "I am John."
    },
    {
      "header": "anything hot",
      "body": "That is a glass."
    }
  ]
}
'

```

## search

```auto
curl -XGET "es:9200/english/_search?pretty" -H 'Content-Type: application/json' -d'
{
  "_source": 'false',
  "size": 20,
  "query": {
    "nested": {
      "path": "contents",
      "score_mode": "max",
      "query": {
          "simple_query_string":{
          "query": "I am",
          "fields": ["contents.header","contents.body"],
          "auto_generate_synonyms_phrase_query": 'true'
        }
      },
      "inner_hits": {
        "size": 1
      }
    }
  }
}
'

```

## result

```auto
{
  "took" : 6,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.4723401,
    "hits" : [
      {
        "_index" : "english",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 1.4723401,
        "inner_hits" : {
          "contents" : {
            "hits" : {
              "total" : 1,
              "max_score" : 1.4723401,
              "hits" : [
                {
                  "_index" : "english",
                  "_type" : "_doc",
                  "_id" : "2",
                  "_nested" : {
                    "field" : "contents",
                    "offset" : 0
                  },
                  "_score" : 1.4723401,
                  "_source" : {
                    "header" : "something special",
                    "body" : "I am John."
                  }
                }
              ]
            }
          }
        }
      },
      {
        "_index" : "english",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.4723401,
        "inner_hits" : {
          "contents" : {
            "hits" : {
              "total" : 1,
              "max_score" : 1.4723401,
              "hits" : [
                {
                  "_index" : "english",
                  "_type" : "_doc",
                  "_id" : "1",
                  "_nested" : {
                    "field" : "contents",
                    "offset" : 0
                  },
                  "_score" : 1.4723401,
                  "_source" : {
                    "header" : "something special",
                    "body" : "I am John."
                  }
                }
              ]
            }
          }
        }
      }
    ]
  }
}

```

In the search response, there is two hits. And two is same content except for `"_id"`.

Then I would like to remove one hit which is similar to another.

If someone have good solution for it, please help me...!!

* * *

The following solution of eliminating duplication "Field Collapsing" doesn't seems to be fit in using nested query.  
[https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-collapse.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-collapse.html)

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [February 25, 2019, 6:52am UTC](https://discuss.elastic.co/t/is-it-possible-to-eliminate-duplication-of-search-response-when-using-nested-query/169750/2 "2019-02-25T06:52:14Z")

</div>

You should may be do that at index time and basically index only one document.

---

<div class="post-metadata">

**Author:** ![Masanori\_Ohnishi](https://avatars.discourse-cdn.com/v4/letter/m/278dde/32.png) [@Masanori\_Ohnishi](https://discuss.elastic.co/u/Masanori_Ohnishi)\
**Post date:** [March 13, 2019, 4:49am UTC](https://discuss.elastic.co/t/is-it-possible-to-eliminate-duplication-of-search-response-when-using-nested-query/169750/3 "2019-03-13T04:49:27Z")

</div>

Thank you for your reply.

I'll try to change way of indexing only one document.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 10, 2019, 4:49am UTC](https://discuss.elastic.co/t/is-it-possible-to-eliminate-duplication-of-search-response-when-using-nested-query/169750/4 "2019-04-10T04:49:29Z")

</div>

Closed on April 10, 2019, 4:49am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.