How to clean up storage space in elasticsearch cluster

Elastic Stack Elasticsearch
1

I'm currently about 80% storage used accross my ES data nodes. I was trying to clear up indexes to get more storage back, but I guess the deleting of indices alone does not remove documents.

Problem 2:
I am looking for a manual temp solution to get disk space back now

Problem 1:
Next I want to delete indices and documents older than 7 days.

2

For this the best thing to do is to create a new index every day and them after 7 days. It is very very inefficient to 1/7 of and index every day. If you delete the index that should give you the space back within a few seconds.

3

Nik,

Thanks for the reply. I am creating indexes every new day, but when I delete the index I am not getting the space back.

Here is how I'm doing the deletion of indices:

curl -XDELETE 'http://172.24.29.X:9200/'

4

I presume you're putting the index name at the end? Even so, it should clear up soon, if not immediately. Keep checking cluster state.

5

Yeah. I could have been more specific.

I am including the index name and that index does get deleted, but the documents and Free Disk Space does not change.

Are you saying after we delete an index the documents will also get removed?

Best,
Monty

6

Yes. You can go to the node where the indices reside and check the indices subdirectory of where your data resides and a df will show they are gone after deletion. Filesystem queue of I/O operations may delay the eventual removal, but you should see that the indices disappear almost instantly.