Remove duplicate results in output of search


  • attached a picture of my current output
  • search quarry.

I am trying to search through all my logs and grab all the src_ips from my Nginx server.

They are located in a field called src_ip.

I am able to get just the src_ips back, but I have a lot of duplicates in my results.

Is there any way to remove the duplicates in the output of the search?

GET /logstash-2019.05.17/_search?pretty=true
  "aggs": {
    "src_ip_dedupe": {
      "cardinality": {
        "field" : "src_ip.keyword"
  "_source": ["src_ip"],
  "query": {
    "exists": {
      "field": "src_ip.keyword"

You're seeing the results of the query. The aggregation results will be elsewhere in the response. Look for the src_ip_dedupe key. The unique IPs will be in that object.
If all you're after it's the aggregation results, add "size: 0" to the request body to stop the hits bring returned as well.
Hope this helps.

thank you!

is there anyway for it not to give me the output of key? (is it possible for it to be src_ip?)


Afraid not. All aggregation types return results in that way.

mmmmmmmmmmmmmmmmmmmmmm that really sucks.

hey elk.... learn from Splunk

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.