Background segment merge looks like dosen't work


(Wang Xiang Ustc) #1

I use delete by query command to delete old data. And I have already look the document and it said that the deleted data is actualy dosen't delete, just tag this data will be delete, and the segment merge will run background, and delete the data be taged. But I delete the old data several days ago, and today I find that the data is not deleted at all.
I already run this command:
PUT /_cluster/settings
{
"persistent" : {
"indices.store.throttle.max_bytes_per_sec" : "100mb"
}
}

Is any other command need to be run?


(Mark Walkom) #2

How do you know the data is not deleted?
Have other things been deleted or updated since then?


(Wang Xiang Ustc) #3

I use the plugin named head, and can see the information of index, and it looks like this:

"primaries": {
"docs": {
"count": 6055,
"deleted": 750
},
"store": {
"size_in_bytes": 3137350,
"throttle_time_in_millis": 0
},

the "deleted" means the data need to be delete. after I run /_forcemerge?max_num_segments=1 command, the "deleted" will be 0, and means the data is really be deleted.


(Mark Walkom) #4

Does it drop to 0 after a force merge?


(Christian Dahlqvist) #5

When you delete a document in Elasticsearch, it is actually an update that creates a tombstone record, and this is reflected as a deleted document in the stats. You will however no longer be able to retrieve this document. Space is however not reclaimed until the immutable segments holding the deleted records are merged. This happens periodically or when you do a force merge down to 1 segment, which leaves no more tombstone records in the segments and the deleted count therefore drops to 0.


(Wang Xiang Ustc) #6

yes, change to 0 after force merge


(Wang Xiang Ustc) #7

yes, I think it will be delete periodically, but after several days it dosen't be clear. Need to run some command ?


(system) #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.