How can I delete documents 3 months older?

Elastic Stack Kibana
1

I have Elasticsearch and Kibana 8.6 and I have an index with a size of 115GB. I would like to query by @timestamp and delete documents older than April 1, 2023. How can I do that? I am new to the query part and not sure what is the best syntax to query and delete.

2

Hi @Mary2022,

You can use the delete_by_query API to delete the documents that match a given query. Using that combined with a range query will allow you to delete documents within a given date range, similar to the below:

POST /my-index/_delete_by_query
{
  "query": {
    "range": {
      "timestamp": {    
        "lte": "2023-004-01T00:00:00", 
      }
    }
  }
}

I would recommend running the query via a basic _search first to make sure you are capturing the results you want before deletion. Hope that helps!

1 Like
3

You are also best off migrating your index approach to use time based ILM. It's far more efficient.

1 Like
4

Absolutely @warkolm! ILM will take care of it for you with regular deletion rather than using delete_by_query for a one off. :smile:

5

Sorry if I am a little lost but does ILM only apply for new documents?

I use the reindex API to reindex only the results/documents obtained from the API to a new index. Ones that id done I will delete the index.

6

It can be applied to older indices as well. Check out this section of the documentation.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.