Background segment merge looks like dosen't work

Elastic Stack Elasticsearch
1

I use delete by query command to delete old data. And I have already look the document and it said that the deleted data is actualy dosen't delete, just tag this data will be delete, and the segment merge will run background, and delete the data be taged. But I delete the old data several days ago, and today I find that the data is not deleted at all.
I already run this command:
PUT /_cluster/settings
{
"persistent" : {
"indices.store.throttle.max_bytes_per_sec" : "100mb"
}
}

Is any other command need to be run?

2

How do you know the data is not deleted?
Have other things been deleted or updated since then?

3

I use the plugin named head, and can see the information of index, and it looks like this:

"primaries": {
"docs": {
"count": 6055,
"deleted": 750
},
"store": {
"size_in_bytes": 3137350,
"throttle_time_in_millis": 0
},

the "deleted" means the data need to be delete. after I run /_forcemerge?max_num_segments=1 command, the "deleted" will be 0, and means the data is really be deleted.

4

Does it drop to 0 after a force merge?

5

When you delete a document in Elasticsearch, it is actually an update that creates a tombstone record, and this is reflected as a deleted document in the stats. You will however no longer be able to retrieve this document. Space is however not reclaimed until the immutable segments holding the deleted records are merged. This happens periodically or when you do a force merge down to 1 segment, which leaves no more tombstone records in the segments and the deleted count therefore drops to 0.

6

yes, change to 0 after force merge

7

yes, I think it will be delete periodically, but after several days it dosen't be clear. Need to run some command ?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.