Freeing up disk space (again)

Elastic Stack Elasticsearch
1

We store our ES 5.2 data on a separate D drive on a Windows platform (syslog01) - and it has become full.

Having Googled and gone through the forum, first I tried deleting some indices using the DELETE method and Postman :

http://syslog01:9200/winlogbeat-2016*

http://syslog01:9200/netflow-2016*

Free disk space reported by the OS remained unchanged.

Then I tried using http://syslog01:9200/_forcemerge?only_expunge_deletes=true (_optimize suggested elsewhere did not work)

We see returned: {"_shards":{"total":833,"successful":417,"failed":0}} - but free disk space reported by the OS still remained unchanged.

Is the 416 difference between "total" and "successful" the 2016 indices deleted above?

Finally, I then installed Python etc and ran curator and removed another 14 indices. Success! Some disk space was freed up - but only that of the 14 indices. Moving forward I now have a solution - so now I have to recover the space from the indices "deleted" previously.

Any suggestions or ideas please?

Thanks

2

What Windows version?

What does the output of _cat/indices show?

3

Thanks for the reply.

Windows Server 2012R2

_cat/indices shows a total of 98 indices made up of:

14 - .monitoring-*
1 - .security (we had X-pack trial, now uninstalled)
1- .kibana
6 - winlogbeat-2017* (from some testing we were doing)
76 - logstash-netflow9-2017*

None of the earlier 2016 indicies that I deleted (see first message) are listed.

The total size (adding up last column) is about 200Gb and Windows now reports 21Gb free on a 220Gb disk - which looks consistent.

Repeating the above forcemerge I also now see {"_shards":{"total":843,"successful":422,"failed":0}} - but of course new indicies are created daily.

The winlogbeat-2017* and logstash-netflow9-2017* each show 5 shards per index - the others just 1 - which makes a total of 426 - not quite 422 - not sure if it ought to be?

I have tried http://syslog01:9200/_cat/allocation/?v

I see:

shards disk.indices disk.used disk.avail disk.total disk.percent host ip node
422 196.6gb 197.4gb 22.5gb 219.9gb 89 10.10.101.83 10.10.101.83 -pdpZDq
421 UNASSIGNED

I am very confused - before I ran the first delete and removed approximately half the indicies we were using 199Gb (as reported by the OS) - now with half as many indices I am still using same disk space!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.