Retrieve duplicate data using Kibana search bar

[mmbtenorio (Marge Tenorio)]

I've been working on retrieving duplicate data from Elasticsearch. However the duplicate filter query I have is not working as expected. I tried the query on KIBANA's search tab but its always showing an error message.

I hope that if my query will work on Kibana, the investigation would be easier. Could you help me create the right query for this?

POST /costing_requests/_search
{   
    "aggs": {
         "duplicateCount": {
              "terms": {
                  "field": "CostInvoiceNumber",
                  "min_doc_count": 2
              }
         }
     },        
    "query": {
      "filtered": {
         "filter": {
            "bool": {
                 "must": [{
                    "range": {
                       "tslice": {
                           "gte": "2017-01-18",
                           "lte": "2017-01-19"
                        }
                    }
                 }],
                "must_not": []
             }
           }
        }
     }
}

[LeeDr (Lee Drengenberg)]

Hi Marge,

I'm not sure if this is what you're looking for, but if I have the following data, and I used the Advanced JSON Input to set the min_doc_count to 100,000 I get these results;

And if I change it to 200,000 I eliminate the results from Feb 8th.

Here's the request generated by Kibana;

{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "*",
            "analyze_wildcard": true
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": 1486052951832,
              "lte": 1486657751832,
              "format": "epoch_millis"
            }
          }
        }
      ],
      "must_not": []
    }
  },
  "_source": {
    "excludes": []
  },
  "aggs": {
    "2": {
      "date_histogram": {
        "field": "@timestamp",
        "interval": "1d",
        "time_zone": "America/Chicago",
        "min_doc_count": 200000
      }
    }
  }
}