Remove duplicate results in output of search

Elastic Stack Elasticsearch
1

Hello,

  • attached a picture of my current output
  • search quarry.

I am trying to search through all my logs and grab all the src_ips from my Nginx server.

They are located in a field called src_ip.

I am able to get just the src_ips back, but I have a lot of duplicates in my results.

Is there any way to remove the duplicates in the output of the search?

GET /logstash-2019.05.17/_search?pretty=true
{
  "aggs": {
    "src_ip_dedupe": {
      "cardinality": {
        "field" : "src_ip.keyword"
      }
    }
  }, 
  "_source": ["src_ip"],
  "query": {
    "exists": {
      "field": "src_ip.keyword"
    }
  }
}

Annotation%202019-05-18%20125209
Annotation 2019-05-18 125209.png455×616 12.3 KB

2

You're seeing the results of the query. The aggregation results will be elsewhere in the response. Look for the src_ip_dedupe key. The unique IPs will be in that object.
If all you're after it's the aggregation results, add "size: 0" to the request body to stop the hits bring returned as well.
Hope this helps.

3

thank you!

is there anyway for it not to give me the output of key? (is it possible for it to be src_ip?)

image

4

Afraid not. All aggregation types return results in that way.

5

mmmmmmmmmmmmmmmmmmmmmm that really sucks.

hey elk.... learn from Splunk

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.