ElasticSearch dedupe quarry/ Aggregation question (I am a noob)

Elastic Stack Elasticsearch
1

sup my dudes,

I am working on a search and I need some ElasticSearch Wisdom.
Goal

  • I am trying to get all the Src_ip's along with, IP_rep, and geoip.asn all in on search with no duplicates from an NGINX server that I am pulling logs from.

My issue is that I get back the data I need, but there is a lot of duplicates, and I was curious if anyone knew of a way to run this search a bit better

search is below (mind you I am still a noob)

GET /logstash-2019.05.17/_search
{
  "aggs": {
    "2": {
      "terms": {
        "field": "src_ip.keyword",
        "size": 5,
        "order": {
          "1": "desc"
        }
      },
      "aggs": {
        "1": {
          "cardinality": {
            "field": "src_ip.keyword"
          }
        },
        "3": {
          "terms": {
            "field": "type.keyword",
            "size": 5,
            "order": {
              "1": "desc"
            }
          },
          "aggs": {
            "1": {
              "cardinality": {
                "field": "src_ip.keyword"
              }
            }
          }
        }
      }
    }
  }
}
{
 "_source": ["src_ip", "beat.name", "ip_rep", "geoip.asn", "type"],
  "query": {
    "exists": {
      "field": "src_ip.keyword"
    }
  }
}
2

This question was also posted here
https://stackoverflow.com/questions/56190531/elasticsearch-dedupe-quarry-help-i-am-a-noob

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.