# Elasticsearch housekeeping

**URL:** <https://discuss.elastic.co/t/elasticsearch-housekeeping/185518>\
**Category:** Elasticsearch\
**Created:** [June 12, 2019, 9:44pm UTC](https://discuss.elastic.co/t/elasticsearch-housekeeping/185518 "2019-06-12T21:44:04Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![dey\_subhro](https://avatars.discourse-cdn.com/v4/letter/d/53a042/32.png) [@dey\_subhro](https://discuss.elastic.co/u/dey_subhro)\
**Post date:** [June 12, 2019, 9:44pm UTC](https://discuss.elastic.co/t/elasticsearch-housekeeping/185518/1 "2019-06-12T21:44:04Z")

</div>

Hi,

I am using Amazon ES, can someone tell me what is the ideal way of automatically deleting the old indexes from the cluster considering Amazon ES does not provide the ILM features ?

I have 25 GB of daily index and I want to keep the data for longer period of time for about 6 months. I have read somewhere that data deletion is not a recommended approach as it involves lot of I/O operation. Then what is the way around if I want to maintain small indexes with less data, deleting the old records ?

Also will there be any impact on the memory if the no. of shards increases while I am deleting the documents older than 30 days daily?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 12, 2019, 9:50pm UTC](https://discuss.elastic.co/t/elasticsearch-housekeeping/185518/2 "2019-06-12T21:50:58Z")

</div>

[Elasticsearch Curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html) should work for you.

You could always upgrade to our Elasticsearch Service as well 🙂

---

<div class="post-metadata">

**Author:** ![dey\_subhro](https://avatars.discourse-cdn.com/v4/letter/d/53a042/32.png) [@dey\_subhro](https://discuss.elastic.co/u/dey_subhro)\
**Post date:** [June 13, 2019, 9:09am UTC](https://discuss.elastic.co/t/elasticsearch-housekeeping/185518/3 "2019-06-13T09:09:01Z")

</div>

HI @warkolm

I think curator will not work in Amazon ES.

Also if you can answer this query mentioned above  
"I have 25 GB of daily index and I want to keep the data for longer period of time for about 6 months. I have read somewhere that data deletion is not a recommended approach as it involves lot of I/O operation. Then what is the way around if I want to maintain small indexes with less data, deleting the old records ?"

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 13, 2019, 2:22pm UTC](https://discuss.elastic.co/t/elasticsearch-housekeeping/185518/4 "2019-06-13T14:22:16Z")

</div>

Curator works just fine with AWS Elasticsearch, so long as you are using AWS Elasticsearch version 5.4 or higher.

---

<div class="post-metadata">

**Author:** ![goodmirek5849](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/goodmirek5849/32/39761_2.png) [@goodmirek5849](https://discuss.elastic.co/u/goodmirek5849)\
**Post date:** [June 19, 2019, 1:50pm UTC](https://discuss.elastic.co/t/elasticsearch-housekeeping/185518/5 "2019-06-19T13:50:01Z")

</div>

> [@warkolm](#):
>
> You could always upgrade to our Elasticsearch Service as well

I would not recommend that. I am using Elastic Cloud and compared to ES running directly on EC2 it is far less reliable, far more expensive. I have to use curator in addition to ILM. Also, ILM shown strange behavior, which nobody was able to explain. For production use, Elastic Cloud and ILM is far less reliable choice than self-hosted or AWS managed ES with curator.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 19, 2019, 10:46pm UTC](https://discuss.elastic.co/t/elasticsearch-housekeeping/185518/6 "2019-06-19T22:46:20Z")

</div>

Welcome to the community 🙂

Are there other threads that you've posted under a different account with these issues that we might be able to take a look at?

---

<div class="post-metadata">

**Author:** ![gabriel\_tessier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabriel_tessier/32/27911_2.png) [@gabriel\_tessier](https://discuss.elastic.co/u/gabriel_tessier)\
**Post date:** [June 20, 2019, 1:00am UTC](https://discuss.elastic.co/t/elasticsearch-housekeeping/185518/7 "2019-06-20T01:00:49Z")

</div>

Hi,

Have a look at this:

> <https://gist.github.com/jarpy/f65b24209e0a9240207599a73ce99bc9>

---

<div class="post-metadata">

**Author:** ![dey\_subhro](https://avatars.discourse-cdn.com/v4/letter/d/53a042/32.png) [@dey\_subhro](https://discuss.elastic.co/u/dey_subhro)\
**Post date:** [June 26, 2019, 2:29pm UTC](https://discuss.elastic.co/t/elasticsearch-housekeeping/185518/8 "2019-06-26T14:29:09Z")

</div>

No @warkolm I haven't posted it anywhere

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 24, 2019, 2:29pm UTC](https://discuss.elastic.co/t/elasticsearch-housekeeping/185518/9 "2019-07-24T14:29:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
