I am thinking about pulling in my Graphite/statd data in to ElasticSearch. Over time I want to remove data to gradually make the index smaller, I was wondering if there was a way that I could write a delete statement to delete all but 1 document per minute.
For example:
The application/statsd might send out 23 lines for the same static within 1 minute. After about a day, I only want to retain 1 record per minute and a week later I want to retain only 1 document every 10 minutes, and so on. A month or year from now I don't care about the 1 minute transactions just a summary of the hour or day so there is no reason to keep the second and minute data.
I know I could write this in a script to query the documents and delete the ones individually but I was hoping to learn of a fancy way of doing this with a ES Query instead.
Thanks for any feed back.