Identify and delete duplicates on several indexes

Elastic Stack Elasticsearch
1

Hello,

I search a lot in the threads but still can't find a solution.
Following to issue with my batch, I got to refeed my elastic but like I use some dated index, the _id param was not sufficient to keep some unicity..

So now, I have duplicates of documents with all the same info and id but on different indexes (which are all on same alias).

After lot of tries, I was able to identify the duplicates I would like remove thx to this request (basically, it return the more recent addition between 2 identic doc).

GET /event-*/Database/_search
{
  "size": "0",
  "aggs": {
    "dedup": {
      "terms": {
        "field": "key",
        "min_doc_count": 2
      },
      "aggs": {
        "dedup_docs": {
          "top_hits": {
            "size": "1",
            "sort": [
              {
                "@timestamp": {
                  "order": "desc"
                }
              }
            ]
          }
        }
      }
    }
  }
}

However, I'm still not able to delete them like multiple index is rejected by DELETE.

I tried to update a field to next delete index by index (i have "only" 20 ones so it's not big deal. but not able anyway to do some correct modifications.

Many thx for your help!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.