# Automate deletion of indices in aws elasticcluster **URL:** \ **Category:** Logstash\ **Created:** [May 18, 2019, 1:09pm UTC](https://discuss.elastic.co/t/automate-deletion-of-indices-in-aws-elasticcluster/181682 "2019-05-18T13:09:13Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Shubham\_Yadav1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubham_yadav1/32/41659_2.png) [@Shubham\_Yadav1](https://discuss.elastic.co/u/Shubham_Yadav1)\ **Post date:** [May 18, 2019, 1:09pm UTC](https://discuss.elastic.co/t/automate-deletion-of-indices-in-aws-elasticcluster/181682/1 "2019-05-18T13:09:13Z")
I have 3 questions: 1. how to automate deletion of indices stored in aws\_es: I am using an aws\_elasticsearch to store data where each index is created everyday based on the time stamp they have. I am using -%{+YYYY.MM.dd} at the end of my index name. how can i delete indices which are older than x days and, i want to automate this process. 2. how does logstash keeps a look at files? I want to perform ondemand indexing where i fetch files in the directory where logstash is running, will those files get indexed as i failed to do so by just creating a file with different timestamp. 3. The files in my directory are created each hour, and i am creating one index each day, will all the data will go into the same index which is created each day? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [May 18, 2019, 4:16pm UTC](https://discuss.elastic.co/t/automate-deletion-of-indices-in-aws-elasticcluster/181682/2 "2019-05-18T16:16:41Z")
This is really an elasticsearch question, not a logstash question. I suggest you read up on [ILM](https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle-management.html) and post any questions you have about it in the elasticsearch forum rather than this one. If you are using daily indexes then yes, all the data for one day, based on the UTC timestamp, goes in to one index. That means that for most of us the data for one day in the local timezone is split across two indexes. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 18, 2019, 6:10pm UTC](https://discuss.elastic.co/t/automate-deletion-of-indices-in-aws-elasticcluster/181682/3 "2019-05-18T18:10:17Z")
ILM is not available on AWS ES, so you probably need to [use Curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html) or switch to [Elastic Cloud](https://www.elastic.co/cloud/elasticsearch-service). ---
**Author:** ![Shubham\_Yadav1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubham_yadav1/32/41659_2.png) [@Shubham\_Yadav1](https://discuss.elastic.co/u/Shubham_Yadav1)\ **Post date:** [May 21, 2019, 10:46am UTC](https://discuss.elastic.co/t/automate-deletion-of-indices-in-aws-elasticcluster/181682/4 "2019-05-21T10:46:02Z")
Thanks for the reply christian but as far as i know Curator doesn't works with aws es, here is the description ([https://www.elastic.co/guide/en/elasticsearch/client/curator/5.1/faq\_aws\_iam.html](https://www.elastic.co/guide/en/elasticsearch/client/curator/5.1/faq_aws_iam.html)) ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [May 21, 2019, 1:06pm UTC](https://discuss.elastic.co/t/automate-deletion-of-indices-in-aws-elasticcluster/181682/5 "2019-05-21T13:06:26Z")
What version of AWS Elasticsearch are you using? It would have to be a very old version for it to not work with Curator. Amazon patched their versions to include the necessary API calls a long time ago. ---
**Author:** ![martinr\_ubi](https://avatars.discourse-cdn.com/v4/letter/m/b5e925/32.png) [@martinr\_ubi](https://discuss.elastic.co/u/martinr_ubi)\ **Post date:** [May 22, 2019, 12:35am UTC](https://discuss.elastic.co/t/automate-deletion-of-indices-in-aws-elasticcluster/181682/6 "2019-05-22T00:35:51Z")
It's even a recommended tool for AWS ESS: [https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/curator.html](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/curator.html) But yeah versions matter. ---
**Author:** ![Shubham\_Yadav1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubham_yadav1/32/41659_2.png) [@Shubham\_Yadav1](https://discuss.elastic.co/u/Shubham_Yadav1)\ **Post date:** [June 2, 2019, 11:22am UTC](https://discuss.elastic.co/t/automate-deletion-of-indices-in-aws-elasticcluster/181682/7 "2019-06-02T11:22:52Z")
Thanks everyone for their reply. I managed to delete the indices using a ruby script. ---
**Author:** ![Rocky\_RK](https://avatars.discourse-cdn.com/v4/letter/r/ed655f/32.png) [@Rocky\_RK](https://discuss.elastic.co/u/Rocky_RK)\ **Post date:** [June 2, 2019, 1:00pm UTC](https://discuss.elastic.co/t/automate-deletion-of-indices-in-aws-elasticcluster/181682/8 "2019-06-02T13:00:55Z")
@Shubham_Yadav1, Would mind sharing the ruby script for posterity on this community. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 30, 2019, 1:01pm UTC](https://discuss.elastic.co/t/automate-deletion-of-indices-in-aws-elasticcluster/181682/9 "2019-06-30T13:01:01Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.