Unable to free disk space after deleting some documents

Elastic Stack Elasticsearch
#1

Hey All,
I deleted few documents using Java API using bulk API. The response was a success and when I check the disk space, I did not see any change.

Later I used force merge to optimize the index as shown below

 ForceMergeResponse forceMergeResponse = client.admin().indices()
                .forceMerge(new ForceMergeRequest(indexName).onlyExpungeDeletes(true).maxNumSegments(1).flush(true)).actionGet();

But still elastic did not free the space.

I tried to trigger optimize request as well but no success.

curl -XPOST 'localhost:9200/indexName/_optimize?only_expunge_deletes=true'

Can someone please help me with this?

#2

Depending on how many you deleted it could still be running. You can check
it with the cat merges API. In general the best way to free up space is to
remove whole indexes. This is why people use daily or weekly indexes. Well,
there are other reasons too, but this is a good reason.

The API call you made will make one large segment which will cause you
trouble down the line if you make updates to documents in that segment
because it will rarely be selected for merging. Generally you shouldn't use
the optimize API unless you are done writing to the index.

#3

I find this interesting topic from the system resource point of view.
Looks like adding the datetime to file name is very important.

#4

@nik9000 The problem with my design is, I have fixed number of index and each index has multiple index type. Also each index has some restriction and each can grow up to some size; let's say 1GB. Periodically I check the size if it is greater than GB knock off the data from oldest index type and recheck the size again. Even after removing all the documents from index type size doesn't change.

#5

Right. It is much more efficient to nuke whole indexes to free up space. So if possible, you should think about reworking your stuff to use indexes for time periods rather than types per time period.

#6

@nik9000, Sure. Will work on that. But is there any temporary fix for my issue :smiley:

#7