# Indices filling up drive

**URL:** <https://discuss.elastic.co/t/indices-filling-up-drive/34090>\
**Category:** Elasticsearch\
**Created:** [November 7, 2015, 8:31pm UTC](https://discuss.elastic.co/t/indices-filling-up-drive/34090 "2015-11-07T20:31:08Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![kartikvelastic](https://avatars.discourse-cdn.com/v4/letter/k/7ba0ec/32.png) [@kartikvelastic](https://discuss.elastic.co/u/kartikvelastic)\
**Post date:** [November 7, 2015, 8:31pm UTC](https://discuss.elastic.co/t/indices-filling-up-drive/34090/1 "2015-11-07T20:31:08Z")

</div>

Team,

logstash.log rotation is working fine and ELK was working fine till the indices started filling the drive, then I stopped getting anything in the Kibana GUI.  
This is the message in elasticsearch.log:  
[2015-11-07 15:20:22,643][WARN][cluster.routing.allocation.decider] [Death's Head] high disk watermark [10%] exceeded on [vt5xzs7lSiuXEBsKcYoV6Q][Death's Head] free: 6gb[8%], shards will be relocated away from this node

Is there any way to rotate the indices(sounds like a dumb question), but what is the solution to this issue. Thanks team.

Regards,

Kartik Vashishta

---

<div class="post-metadata">

**Author:** ![niraj\_kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niraj_kumar/32/4130_2.png) [@niraj\_kumar](https://discuss.elastic.co/u/niraj_kumar)\
**Post date:** [November 7, 2015, 9:17pm UTC](https://discuss.elastic.co/t/indices-filling-up-drive/34090/2 "2015-11-07T21:17:25Z")

</div>

You can use a tool named curator to delete indices that are 'n' days old.

---

<div class="post-metadata">

**Author:** ![kartikvelastic](https://avatars.discourse-cdn.com/v4/letter/k/7ba0ec/32.png) [@kartikvelastic](https://discuss.elastic.co/u/kartikvelastic)\
**Post date:** [November 7, 2015, 10:08pm UTC](https://discuss.elastic.co/t/indices-filling-up-drive/34090/3 "2015-11-07T22:08:32Z")

</div>

[root@test3 elasticsearch]# curator delete indices --older-than 2 --time-unit days --timestring 2015.11.07  
2015-11-07 17:08:07,338 INFO Job starting: delete indices  
Traceback (most recent call last):  
File "/usr/bin/curator", line 11, in   
sys.exit(main())  
File "/usr/lib/python2.7/site-packages/curator/curator.py", line 5, in main  
cli( obj={ "filters": [] } )  
File "/usr/lib/python2.7/site-packages/click/core.py", line 700, in **call**  
return self.main(\*args, \*\*kwargs)  
File "/usr/lib/python2.7/site-packages/click/core.py", line 680, in main  
rv = self.invoke(ctx)  
File "/usr/lib/python2.7/site-packages/click/core.py", line 1027, in invoke  
return \_process\_result(sub\_ctx.command.invoke(sub\_ctx))  
File "/usr/lib/python2.7/site-packages/click/core.py", line 1027, in invoke  
return \_process\_result(sub\_ctx.command.invoke(sub\_ctx))  
File "/usr/lib/python2.7/site-packages/click/core.py", line 873, in invoke  
return ctx.invoke(self.callback, \*\*ctx.params)  
File "/usr/lib/python2.7/site-packages/click/core.py", line 508, in invoke  
return callback(\*args, \*\*kwargs)  
File "/usr/lib/python2.7/site-packages/click/decorators.py", line 16, in new\_func  
return f(get\_current\_context(), \*args, \*\*kwargs)  
File "/usr/lib/python2.7/site-packages/curator/cli/index\_selection.py", line 95, in indices  
working\_list = apply\_filter(working\_list, \*\*f)  
File "/usr/lib/python2.7/site-packages/curator/api/filter.py", line 114, in apply\_filter  
p = re.compile(pattern)  
File "/usr/lib64/python2.7/re.py", line 190, in compile  
return \_compile(pattern, flags)  
File "/usr/lib64/python2.7/re.py", line 242, in \_compile  
raise error, v # invalid expression  
sre\_constants.error: bogus escape: '\2'  
[root@test3 elasticsearch]#

---

<div class="post-metadata">

**Author:** ![kartikvelastic](https://avatars.discourse-cdn.com/v4/letter/k/7ba0ec/32.png) [@kartikvelastic](https://discuss.elastic.co/u/kartikvelastic)\
**Post date:** [November 7, 2015, 10:20pm UTC](https://discuss.elastic.co/t/indices-filling-up-drive/34090/4 "2015-11-07T22:20:46Z")

</div>

So, I did this:  
curator delete indices --regex ^logstash-.\*  
and restarted elasticsearch but now I do not see anything in Kibana, even though, elasticsearch log shows this:

[2015-11-07 17:16:38,318][INFO][node] [Crusher] started  
[2015-11-07 17:16:38,752][INFO][gateway] [Crusher] recovered [2] indices into cluster\_state  
[2015-11-07 17:16:38,752][INFO][cluster.service] [Crusher] added {[logstash-test3.kartikv.com-818-11630][X15apYWdRlGRqgcJxOGjvA][[test3.kartikv.com](http://test3.kartikv.com)][inet[/192.168.1.51:9301]]{data=false, client=true},}, reason: zen-disco-receive(join from node[[logstash-test3.kartikv.com-818-11630][X15apYWdRlGRqgcJxOGjvA][[test3.kartikv.com](http://test3.kartikv.com)][inet[/192.168.1.51:9301]]{data=false, client=true}])

---

<div class="post-metadata">

**Author:** ![kartikvelastic](https://avatars.discourse-cdn.com/v4/letter/k/7ba0ec/32.png) [@kartikvelastic](https://discuss.elastic.co/u/kartikvelastic)\
**Post date:** [November 7, 2015, 10:30pm UTC](https://discuss.elastic.co/t/indices-filling-up-drive/34090/5 "2015-11-07T22:30:35Z")

</div>

a system reboot fixed it. I'd like to be able to fix this w/o a reboot. This is what was in logstash logs:  
{:timestamp=\>"2015-11-07T17:23:52.769000-0500", :message=\>["INFLIGHT\_EVENTS\_REPORT", "2015-11-07T17:23:52-05:00", {"input\_to\_filter"=\>20, "filter\_to\_output"=\>20, "outputs"=\>[]}], :level=\>:warn}

---

<div class="post-metadata">

**Author:** ![kartikvelastic](https://avatars.discourse-cdn.com/v4/letter/k/7ba0ec/32.png) [@kartikvelastic](https://discuss.elastic.co/u/kartikvelastic)\
**Post date:** [November 8, 2015, 7:41pm UTC](https://discuss.elastic.co/t/indices-filling-up-drive/34090/6 "2015-11-08T19:41:12Z")

</div>

So I did not read the curator examples and documentation as well as I ought to have, I put this in a a script and put it in cron:

#!/bin/bash  
curator --host localhost delete indices --older-than 4 --timestring '%Y.%m.%d' --time-unit days --prefix logstash

---

<div class="post-metadata">

**Author:** ![niraj\_kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niraj_kumar/32/4130_2.png) [@niraj\_kumar](https://discuss.elastic.co/u/niraj_kumar)\
**Post date:** [November 12, 2015, 9:07pm UTC](https://discuss.elastic.co/t/indices-filling-up-drive/34090/7 "2015-11-12T21:07:16Z")

</div>

So did this work for you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:38pm UTC](https://discuss.elastic.co/t/indices-filling-up-drive/34090/8 "2017-07-05T23:38:50Z")

</div>


