ElasticSearch - docs deleted count reduced

Elastic Stack Elasticsearch
1

Hi

For debugging I have checked one of currently writing indices shows the following results

4 days before the following response i got

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open order-feed--201908 ApSD8D7tSb-zC1bb92Vzrg 2 1 39864412 117962312 235.9gb 122gb

I have checked after 4 days , the recent response shows,

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open order-feed--201908 ApSD8D7tSb-zC1bb92Vzrg 2 1 42360806 33008678 108gb 57gb

I would like to know the behavior of docs.deleted.

I Didn't call any force_segments API or any other things on that index. Only update API called on that index.

How the docs.deleted count is reduced in this case. Any help would be appreciated.

2

merging happens automatically in the background and will change the number of deleted documents by reducing the number of segments and removing deleted document during that process.

3

Hi Alexandar,

Thanks for the reply, Understands merging happens automatically.

So the default merging segment size is 5 GB, the indexing process was completed in that index.

So far few segments merged with size to 5.0 GB. Other segments are not merged upto 5 GB(max segment size).

I would like to understand

Why the other segments are not merged upto 5GB, Even though no indexing happened at the index/cluster?

How the floor segments work with the index.

Can we define the max/min no of segments count for index.

FYI, We are running separate force_merge api for older indices to make the segment 1 per shard.

4

in order to decide when merging needs to happen and for which segments, lucene uses a so called merge policy. You may want to read about it in lucene, and also take a look at the https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/index/EsTieredMergePolicy.java policy used in elasticsearch and follow the links in there.

5

Thanks Alexander , Sounds good . Let me have a deep look on this.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.