# How to search for duplicates

**URL:** <https://discuss.elastic.co/t/how-to-search-for-duplicates/232203>\
**Category:** Elasticsearch\
**Created:** [May 12, 2020, 10:39am UTC](https://discuss.elastic.co/t/how-to-search-for-duplicates/232203 "2020-05-12T10:39:28Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![howz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/howz/32/68143_2.png) [@howz](https://discuss.elastic.co/u/howz)\
**Post date:** [May 12, 2020, 10:39am UTC](https://discuss.elastic.co/t/how-to-search-for-duplicates/232203/1 "2020-05-12T10:39:28Z")

</div>

Trying to find duplicate MAC addresses in customer records, what am I doing wrong?

```auto
GET /_search
{
  "query": {
    "match_all": {}
  },
  "aggs": {
    "device_mac": {
      "terms": {
        "field": "device_mac",
        "min_doc_count": 2
      }
    }
  }
}  

```

---

<div class="post-metadata">

**Author:** ![thomasneirynck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thomasneirynck/32/23313_2.png) [@thomasneirynck](https://discuss.elastic.co/u/thomasneirynck)\
**Post date:** [May 12, 2020, 8:18pm UTC](https://discuss.elastic.co/t/how-to-search-for-duplicates/232203/2 "2020-05-12T20:18:14Z")

</div>

that looks about right: [https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#\_minimum\_document\_count\_4](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_minimum_document_count_4)

what's not working?

This seems more of an Elasticsearch question than a Kibana one though.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 16, 2020, 3:55pm UTC](https://discuss.elastic.co/t/how-to-search-for-duplicates/232203/4 "2020-05-16T15:55:50Z")

</div>

How many shards is this data distributed across? The min\_doc\_count is applied once the shards have returned results, so if the duplicates reside in different shards and are few they could be missed as the relevant data may not be returned from all shards. This is described in [the docs](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/search-aggregations-bucket-terms-aggregation.html#_minimum_document_count_4).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 13, 2020, 3:55pm UTC](https://discuss.elastic.co/t/how-to-search-for-duplicates/232203/5 "2020-06-13T15:55:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.