Delete_by_query increases the storage size

Elastic Stack Elasticsearch
1

Hi All,

I have used following "delete_by_query" to delete large number of documents from an index of my ES.

POST /winlogbeat-7.12.0-2021.05.20-000029/_delete_by_query
{
  "query": {
    "match": {
      "event.code": "5157"
    }
  }
}

The first thing I noticed is, the delete operation is taking long time to delete the documents.

Apart from that I have noticed, the size of the index is increasing though the document count reduces.
The intention of the delete operation was to free the storage by removing less important logs.
Can someone please let me know why this is happening and guide me to free the space correctly.

Thanks
Billz1026

2

Elasticsearch shards use immutable segments for storing data so when you delete a document you create a tombstone record, but the documents are not physically deleted until the underlying segments are merged.

3

Hi Christian,

Thanks for the reply. after bit reading I found that I have to run "forcemerge" to free the space. Can you please let me know whether I am in the correct path?

BR,
Billz1026

4

Yes that is correct.

5

Thanks Christian

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.