Remove duplicate results in output of search

[iukea (jordan)]

Hello,

• attached a picture of my current output
• search quarry.

I am trying to search through all my logs and grab all the src_ips from my Nginx server.

They are located in a field called src_ip.

I am able to get just the src_ips back, but I have a lot of duplicates in my results.

Is there any way to remove the duplicates in the output of the search?

GET /logstash-2019.05.17/_search?pretty=true
{
  "aggs": {
    "src_ip_dedupe": {
      "cardinality": {
        "field" : "src_ip.keyword"
      }
    }
  }, 
  "_source": ["src_ip"],
  "query": {
    "exists": {
      "field": "src_ip.keyword"
    }
  }
}

[whatgeorgemade (George Bridgeman | georgebridgeman.com)]

You're seeing the results of the query. The aggregation results will be elsewhere in the response. Look for the src_ip_dedupe key. The unique IPs will be in that object.
If all you're after it's the aggregation results, add "size: 0" to the request body to stop the hits bring returned as well.
Hope this helps.

[iukea (jordan)]

thank you!

is there anyway for it not to give me the output of key? (is it possible for it to be src_ip?)

[whatgeorgemade (George Bridgeman | georgebridgeman.com)]

Afraid not. All aggregation types return results in that way.

[iukea (jordan)]

mmmmmmmmmmmmmmmmmmmmmm that really sucks.

hey elk.... learn from Splunk