# Remove duplicate results in output of search

**URL:** https://discuss.elastic.co/t/remove-duplicate-results-in-output-of-search/181691
**Category:** Elasticsearch
**Created:** [May 18, 2019, 5:53pm UTC](https://discuss.elastic.co/t/remove-duplicate-results-in-output-of-search/181691 "2019-05-18T17:53:57Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![iukea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iukea/32/49083_2.png) [@iukea](https://discuss.elastic.co/u/iukea)
#### Post date: [May 18, 2019, 5:53pm UTC](https://discuss.elastic.co/t/remove-duplicate-results-in-output-of-search/181691/1 "2019-05-18T17:53:57Z")

</div>

Hello,

- attached a picture of my current output
- search quarry.

I am trying to search through all my logs and grab all the src\_ips from my Nginx server.

They are located in a field called src\_ip.

I am able to get just the src\_ips back, but I have a lot of duplicates in my results.

**Is there any way to remove the duplicates in the output of the search?**

```
GET /logstash-2019.05.17/_search?pretty=true
{
  "aggs": {
    "src_ip_dedupe": {
      "cardinality": {
        "field" : "src_ip.keyword"
      }
    }
  }, 
  "_source": ["src_ip"],
  "query": {
    "exists": {
      "field": "src_ip.keyword"
    }
  }
}

```

 ![Annotation%202019-05-18%20125209](https://us1.discourse-cdn.com/elastic/original/3X/8/c/8c4d0b446dffdf79f9f64b2a4513347e95d73de5.png)

---

<div class="post-metadata">

### Author: ![whatgeorgemade](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/whatgeorgemade/32/103246_2.png) [@whatgeorgemade](https://discuss.elastic.co/u/whatgeorgemade)
#### Post date: [May 18, 2019, 7:29pm UTC](https://discuss.elastic.co/t/remove-duplicate-results-in-output-of-search/181691/2 "2019-05-18T19:29:40Z")

</div>

You're seeing the results of the query. The aggregation results will be elsewhere in the response. Look for the `src_ip_dedupe` key. The unique IPs will be in that object.  
If all you're after it's the aggregation results, add `"size: 0"` to the request body to stop the hits bring returned as well.  
Hope this helps.

---

<div class="post-metadata">

### Author: ![iukea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iukea/32/49083_2.png) [@iukea](https://discuss.elastic.co/u/iukea)
#### Post date: [May 19, 2019, 1:19am UTC](https://discuss.elastic.co/t/remove-duplicate-results-in-output-of-search/181691/3 "2019-05-19T01:19:34Z")

</div>

thank you!

is there anyway for it not to give me the output of key? (is it possible for it to be src\_ip?)

![image](https://us1.discourse-cdn.com/elastic/original/3X/c/4/c44c49265a9379e5e7ab78eba84e57b3f56ae23d.png)

---

<div class="post-metadata">

### Author: ![whatgeorgemade](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/whatgeorgemade/32/103246_2.png) [@whatgeorgemade](https://discuss.elastic.co/u/whatgeorgemade)
#### Post date: [May 19, 2019, 6:59am UTC](https://discuss.elastic.co/t/remove-duplicate-results-in-output-of-search/181691/4 "2019-05-19T06:59:23Z")

</div>

Afraid not. All aggregation types return results in that way.

---

<div class="post-metadata">

### Author: ![iukea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iukea/32/49083_2.png) [@iukea](https://discuss.elastic.co/u/iukea)
#### Post date: [May 20, 2019, 3:58pm UTC](https://discuss.elastic.co/t/remove-duplicate-results-in-output-of-search/181691/5 "2019-05-20T15:58:23Z")

</div>

mmmmmmmmmmmmmmmmmmmmmm that really sucks.

hey elk.... learn from Splunk

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 17, 2019, 3:58pm UTC](https://discuss.elastic.co/t/remove-duplicate-results-in-output-of-search/181691/6 "2019-06-17T15:58:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.