# ElasticSearch dedupe quarry/ Aggregation question (I am a noob)

**URL:** <https://discuss.elastic.co/t/elasticsearch-dedupe-quarry-aggregation-question-i-am-a-noob/181628>\
**Category:** Elasticsearch\
**Created:** [May 17, 2019, 4:37pm UTC](https://discuss.elastic.co/t/elasticsearch-dedupe-quarry-aggregation-question-i-am-a-noob/181628 "2019-05-17T16:37:23Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![iukea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iukea/32/49083_2.png) [@iukea](https://discuss.elastic.co/u/iukea)\
**Post date:** [May 17, 2019, 4:37pm UTC](https://discuss.elastic.co/t/elasticsearch-dedupe-quarry-aggregation-question-i-am-a-noob/181628/1 "2019-05-17T16:37:23Z")

</div>

sup my dudes,

I am working on a search and I need some ElasticSearch Wisdom.  
_Goal_

- I am trying to get all the Src\_ip's along with, IP\_rep, and geoip.asn all in on search with no duplicates from an NGINX server that I am pulling logs from.

My issue is that I get back the data I need, but there is a lot of duplicates, and I was curious if anyone knew of a way to run this search a bit better

search is below (mind you I am still a noob)

```
GET /logstash-2019.05.17/_search
{
  "aggs": {
    "2": {
      "terms": {
        "field": "src_ip.keyword",
        "size": 5,
        "order": {
          "1": "desc"
        }
      },
      "aggs": {
        "1": {
          "cardinality": {
            "field": "src_ip.keyword"
          }
        },
        "3": {
          "terms": {
            "field": "type.keyword",
            "size": 5,
            "order": {
              "1": "desc"
            }
          },
          "aggs": {
            "1": {
              "cardinality": {
                "field": "src_ip.keyword"
              }
            }
          }
        }
      }
    }
  }
}
{
 "_source": ["src_ip", "beat.name", "ip_rep", "geoip.asn", "type"],
  "query": {
    "exists": {
      "field": "src_ip.keyword"
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![iukea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iukea/32/49083_2.png) [@iukea](https://discuss.elastic.co/u/iukea)\
**Post date:** [May 17, 2019, 4:50pm UTC](https://discuss.elastic.co/t/elasticsearch-dedupe-quarry-aggregation-question-i-am-a-noob/181628/2 "2019-05-17T16:50:21Z")

</div>

> [@iukea](#):
>
> GET /logstash-2019.05.17/\_search { "aggs": { "2": { "terms": { "field": "src\_ip.keyword", "size": 5, "order": { "1": "desc" } }, "aggs": { "1": { "cardinality": { "field": "src\_ip.keyword" } }, "3": { "terms": { "field": "type.keyword", "size": 5, "order": { "1": "desc" } }, "aggs": { "1": { "cardinality": { "field": "src\_ip.keyword" } } } } } } } } { "\_source": ["src\_ip", "beat.name", "ip\_rep", "geoip.asn", "type"], "query": { "exists": { "field": "src\_ip.keyword" } } }

This question was also posted here  
[https://stackoverflow.com/questions/56190531/elasticsearch-dedupe-quarry-help-i-am-a-noob](https://stackoverflow.com/questions/56190531/elasticsearch-dedupe-quarry-help-i-am-a-noob)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 14, 2019, 4:50pm UTC](https://discuss.elastic.co/t/elasticsearch-dedupe-quarry-aggregation-question-i-am-a-noob/181628/3 "2019-06-14T16:50:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.