Automatically delete 1 month old documents(without deleting index) in elastic search


(Nikhil Kumar) #1

I am new to Elastic Search & I need to delete documents older than 1 month (in all indices) in the elastic search .
I know how to delete a single document by id / deleting an index , but not able to figure out how to automatically delete 1 month old data in elastic search
Can somebody please tell how to do that .
Thanks


(David Pilato) #2

You should use time based indices and just drop the old indices using curator project.

Otherwise have a look at the delete by query API.


(Nikhil Kumar) #3

When i try to create time based index for filebeats,it says "unable to fetch mapping , do you have indices matching that pattern". However the normal index is getting created when i type in filebeat-*.
Can u please tell what's wrong here!

Also can you please provide steps /example of using curator. I am very new to Elastic search.
Thanks


(David Pilato) #4

For the 1st question, I don't know. May be ask in #kibana for help.
Check that indices exist:

GET _cat/indices?v 

Will give the list of existing indices.

For the 2nd question, please read the documentation : https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html