How to delete elasticsearch data which is older than 90 days from a static Index

(Vikas Gopal) #1

Hi Experts,

I have one static Index(I mean I do not create index every day) , but data is keep on coming on daily basis . My requirement is to delete old data from this single index to make more disk space .

I found ES Curator is the solution , but I am afraid of using it , as document says it is only best fit to delete an indices older than mentioned time frame, so as per my understanding if I use curator it will delete my overall index rather just old data sorry if I misunderstood it , please correct me.

So if not curator what other option I have ?


(Magnus Bäck) #2

You can use the delete by query API (which is provided via a plugin as of ES 2.0), but you really really shouldn't. For time-series data the best option is to use time-series indexes.

(Vikas Gopal) #3

So Curator is not an option for me ? actually i am on ES 1.7.1.

(Magnus Bäck) #4

I don't think Curator supports the delete by query API. If it does you should be okay.

(Mark Walkom) #5

Curator does not support DQB.

You are much better off moving to time based indices.

(Vikas Gopal) #6

Hmmmm, what about TTL ?

(Magnus Bäck) #7

Possible but also a bad idea (and deprecated since ES 2.0). Deleting documents from big shards (regardless of whether via TTL or other method of deleting documents) is expensive.

(system) #8