How to DELETE logs from elasticsearch index which are older then 90 days?

Elastic Stack Elasticsearch
1

Is it possible to delete logs from elasticsearch index which are older than 3 months?

NOTE: I don't want to create daily based or monthly based index and but want only one index and want to delete 3-month-old data inside that index based on timestamps.

Does anybody have any idea how to delete data after three months automatically?

Thanks in advance.

2

As long as you know this is really inefficient and absolutely not what is recommended by elastic, you can do whatever you want. :blush:
I just want to make sure you know what you are doing.

Does anybody have any idea how to delete data after three months?

Use the Delete By Query API: Delete by query API | Elasticsearch Guide [8.11] | Elastic

Does anybody have any idea how to delete data after three months automatically?

Add this query in a crontab or something like this to run it everyday for example.

1 Like
3

Hi @dadoonet,

Thank you so much for replying. It's deleting the data as I want but I'm doing hard code the the timestemp column as given below.

POST metricbeat-system-monitor/_delete_by_query
{
"query": {
"range" : {
"@timestamp" : {
"gte" : "01/27/2019",
"lte" : "01/29/2019",
"format": "MM/dd/yyyy||yyyy"
}
}
}
}

@dadoonet, Could you please suggest me the better way to maintain the 3-month historical data so that I can compare historical data to current data.

4

Is it a question?

Not sure I understood the question.
To manage time series data with a retention period the solution is to use time based indices. But you said that you don't want this.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.